{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "collapsed_sections": [],
      "authorship_tag": "ABX9TyMAuQtEtatMLnjgs5oANTkA",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/dlsyscourse/public_notebooks/blob/main/17_generative_adversarial_networks_implementation.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Mpn1ti5Urdsv"
      },
      "source": [
        "# Lecture 17: Generative adversarial networks implementation\n",
        "\n",
        "In this lecture, we are going to implement a version of generative adversarial training.\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qXysoqn-vZuF"
      },
      "source": [
        "## Prepare the codebase\n",
        "\n",
        "To get started, please clone a version of needle repo from the github. You should be able to use the need repo after finishng HW2"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "JjEIRTyr8ajf",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "f459c270-8644-4eaa-be48-5bc265b32879"
      },
      "source": [
        "# Code to set up the assignment\n",
        "from google.colab import drive\n",
        "drive.mount('/content/drive')\n",
        "%cd /content/drive/MyDrive/\n",
        "!mkdir -p 10714f22\n",
        "%cd /content/drive/MyDrive/10714f22\n",
        "# comment out the following line if you run it for the second time\n",
        "# as you already have a local copy of code\n",
        "# !git clone https://github.com/myrepo/needle lecture17\n",
        "!ln -s /content/drive/MyDrive/10714f22/lecture17 /content/needle"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Drive already mounted at /content/drive; to attempt to forcibly remount, call drive.mount(\"/content/drive\", force_remount=True).\n",
            "/content/drive/MyDrive\n",
            "/content/drive/MyDrive/10714f22\n",
            "Cloning into 'lecture17'...\n",
            "remote: Enumerating objects: 917, done.\u001b[K\n",
            "remote: Counting objects: 100% (184/184), done.\u001b[K\n",
            "remote: Compressing objects: 100% (115/115), done.\u001b[K\n",
            "remote: Total 917 (delta 104), reused 122 (delta 68), pack-reused 733\u001b[K\n",
            "Receiving objects: 100% (917/917), 265.21 KiB | 1.99 MiB/s, done.\n",
            "Resolving deltas: 100% (531/531), done.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Xe3vClsD9jlq",
        "outputId": "ede8f7b5-3e3b-45de-e52b-590fe67e6f38"
      },
      "source": [
        "!python3 -m pip install pybind11"
      ],
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n",
            "Collecting pybind11\n",
            "  Downloading pybind11-2.10.1-py3-none-any.whl (216 kB)\n",
            "\u001b[K     |████████████████████████████████| 216 kB 5.3 MB/s \n",
            "\u001b[?25hInstalling collected packages: pybind11\n",
            "Successfully installed pybind11-2.10.1\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DFxG3p3S1sBq"
      },
      "source": [
        "We can then run the following command to make the path to the package available in colab's environment as well as the PYTHONPATH."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bix8OXLuCOKt",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "a8ccf8ba-ff33-441e-ddd3-4697dd625b5c"
      },
      "source": [
        "%set_env PYTHONPATH /content/needle/python:/env/python\n",
        "import sys\n",
        "sys.path.append(\"/content/needle/python\")"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "env: PYTHONPATH=/content/needle/python:/env/python\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BBIuE2jc1DaU"
      },
      "source": [
        "## Components of a generative advesarial network\n",
        "\n",
        "There are two main components in a generative adversarial network\n",
        "- A generator $G$ that takes a random vector $z$ and maps it to a generated(fake) data $G(z)$.\n",
        "- A discriminator that attempts to tell the difference between the real dataset and the fake one.\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "T-1piPnmkMVg"
      },
      "source": [
        "![image.png]()"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_mBCkBtSa54H"
      },
      "source": [
        "import needle as ndl\n",
        "import numpy as np\n",
        "from needle import nn\n",
        "from matplotlib import pyplot as plt"
      ],
      "execution_count": 6,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cbT1kk_4k6HE"
      },
      "source": [
        "### Parpare the training dataset\n",
        "\n",
        "For demonstration purpose, we create our \"real\" dataset as a two dimensional gaussian distribution. \n",
        "\n",
        "\\begin{equation} \n",
        "X \\sim \\mathcal{N}(u, \\Sigma), \\Sigma = A^T A \n",
        "\\end{equation}"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qrIYM_PDbhFm"
      },
      "source": [
        "A = np.array([[1, 2], [-0.2, 0.5]])\n",
        "mu = np.array([2, 1])\n",
        "# total number of sample data to generated\n",
        "num_sample = 3200\n",
        "data = np.random.normal(0, 1, (num_sample, 2)) @ A + mu"
      ],
      "execution_count": 7,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 282
        },
        "id": "3yDlolmEizvN",
        "outputId": "d1854b12-26ff-4b51-f0ea-b1630b1e992f"
      },
      "source": [
        "plt.scatter(data[:,0], data[:,1], color=\"blue\", label=\"real data\")\n",
        "plt.legend()"
      ],
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.legend.Legend at 0x7fc63ddf0990>"
            ]
          },
          "metadata": {},
          "execution_count": 8
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXIAAAD4CAYAAADxeG0DAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3df5Bc1XUn8O+Z1oxFS3KMWtqYRZoevBsIIGO0Gn7I2nWwYTEZbMNu2UXikQBDrDAyLu06Wyko/QFbqanaWlJ25EqELIOwrOmKCzvWOk5IEGQNcTDgDFjY/DJQRkIjs9ZMywYNAqTRnP3jzlN3v3n39Xv93uv3Xvf3U9U1mp6e13dm4PTtc8+9R1QVRESUXz1pD4CIiKJhICciyjkGciKinGMgJyLKOQZyIqKcW5DGky5btkwHBgbSeGoiotx66qmnplR1ufv+VAL5wMAAxsfH03hqIqLcEpEDXvcztUJElHMM5EREOcdATkSUc6nkyL2cOHECExMTeOedd9IeSi4tXLgQK1asQG9vb9pDIaI2y0wgn5iYwJIlSzAwMAARSXs4uaKqqFarmJiYwFlnnZX2cIiozTKTWnnnnXdQKpUYxFsgIiiVSnw3Q5SCSgUYGAB6eszHSqX9Y8jMjBwAg3gE/N0RtV+lAmzcCBw7Zj4/cMB8DgDDw+0bR2Zm5EREebNlSy2IO44dM/e3EwN5jAYGBjA1NeX7mG984xu49dZbfR/zyCOP4Ec/+lGcQyOiBLz2Wrj7k8JA7kFVMTs7m9rzM5AT5UN/f7j7k5LbQB73AsP+/ftxzjnn4Prrr8eqVatw8OBB3HXXXbjoootwwQUX4I477jj12GuvvRZr1qzB+eefjx07djS99n333Yezzz4bF198MR577LFT93//+9/HJZdcgtWrV+OKK67Ar371K+zfvx/bt2/HV77yFVx44YX44Q9/6Pk4Ikrf6ChQLDbeVyya+9tKVdt+W7Nmjbo9//zz8+6zGRtTLRZVgdqtWDT3t+rVV19VEdHHH39cVVUffPBB/fznP6+zs7N68uRJvfrqq/XRRx9VVdVqtaqqqseOHdPzzz9fp6amVFW1XC7r5ORkw3V/+ctf6sqVK/Xw4cP67rvv6oc//GH9whe+oKqqR44c0dnZWVVV/frXv65f+tKXVFX1jjvu0LvuuuvUNWyPcwvzOySieIyNqZbLqiLmY5Q41AyAcfWIqZmqWgnKb4EhykpxuVzGpZdeCgDYu3cv9u7di9WrVwMApqen8fLLL+MjH/kIvvrVr2LPnj0AgIMHD+Lll19GqVTyvOaTTz6Jyy67DMuXmwPLrrvuOrz00ksATO38ddddh9dffx3Hjx+31oAHfRwRtd/wcHsrVLzkMrWS1ALDokWLTv1bVXH77bdj37592LdvH1555RXcfPPNeOSRR/Dwww/j8ccfxzPPPIPVq1e3XL/9xS9+Ebfeeit+9rOf4Wtf+5r1OkEfR0TdKZeBvB0LDB//+Mexc+dOTE9PAwAOHTqEw4cP44033sDpp5+OYrGIF198EU888YTvdS655BI8+uijqFarOHHiBL797W+f+tobb7yBM888EwCwa9euU/cvWbIER48ebfo4IiIgpkAuIv9dRJ4TkWdF5K9FZGEc17VpxwLDlVdeic9+9rNYu3YtPvjBD+LTn/40jh49iquuugozMzM499xzcdttt51KxdicccYZuPPOO7F27VqsW7cO55577qmv3XnnnfjMZz6DNWvWYNmyZafu/+QnP4k9e/acWuy0PY6I4pOFHZot80qch7kBOBPAqwBOm/v8fgA3+n1P1MVO1fYuMOQFFzuJWpNEAUUSYFnsjCu1sgDAaSKyAEARwC9juq7V8DCwfz8wO2s+pr3YQETpijKjzsoOzVZFDuSqegjAnwN4DcDrAN5Q1b3ux4nIRhEZF5HxycnJqE9LRHSKc+bJgQNmPu2ceRI0mGdlh2arIgdyETkdwDUAzgLwbwEsEpH17sep6g5VHVTVQacUz+MxUYfTtfi7o27Wyoy6fgbfY4mE7d6h2ao4UitXAHhVVSdV9QSA7wL4cNiLLFy4ENVqlQGpBTp3HvnChYmuMRNlVtgZtXsGf/Lk/MekskOzRXFsCHoNwKUiUgTwNoDLAYyHvciKFSswMTEBpl1a43QIIupG/f0mKHvd78VrBg8AhYJZd+vvN0E8L2tvkQO5qj4pIt8B8DSAGQA/AdD8ABKX3t5e7lgkopaMjjaeCw74z6htM/XZWXPLm1iqVlT1DlX9XVVdpaobVPXdOK5LRBTE8DCwYwdQLgMi5uOOHfYZdVZOLYxLLnd2EhG5hSlJzsyphTFhICeizGjX7sqwM/iskzSqRAYHB3V8PPR6KBF1MHf/S0epBGzdmt8gGycReUpVB933c0ZORJlgqySpVsNt7rHJ9VkqTTCQE1Em+O2ijLpdPurOzzgk+ULCQE5EmdCsYiTKdvm0z1JJ+oWEgZyIMsGrkqRelNLAtM9SSfqFhIGciDLBqSTx6poYtTQw7brxpF9IGMiJKDOGh4GpKWBsLN7SQK/Zfm8vMD3dnsXPpF9IGMiJKHP8Nve0smjorhsvlczHanV+zjqJRcnENyB5dZtI+ubVIYiIqBmvTj4i5mOYTmHlcuM1nFupFL1TkK17WRxdzWDpEMRATkS5YQvAtqBrC55O8A96K5ebX9P5WpIt4xjIiaitkuirGyQAO0HXL6g2e0Fw30SaX1PVft36F4IoGMiJqG3inJnWvyAUCsECr6p/ULWNr1TyD8TNArXthcZ5IYjKFsi52ElEsYurbjpIJx+3QsF89Cv5sx2atXWr/6JkszLCtMocGciJKHa2gHfgQLiKEL9OPjZOsLcFT1Xz3MD8yphmpyI2C9RpHY/LQE5EsbMFPJFw29RtLwgnT/oH82XLgKEh+05Rv+d2lz4CtRef6WlTf+7+mYaGat+byvG4XvmWpG/MkRN1Nr8ywTALgbacdZBbsah63nn+j3Hy5WGqUBYs8H6uuCpT/MCSI4/lPHIReR+AewCsAqAAblLVx22P53nkRJ3viiuAf/qnYI8tlczmHMDMfGdnzWx2agp4663kxgiYGfaJE42f33efmUUPDHg3dfZSLtdm8ElJ+jzyrQD+UVV/F8CHALwQ03WJKAZhdytG3d24aVPwIA7UgjhQa3584EDyQVykMYgD5vMNG8zPHOYslHYdwOUl8oxcRH4LwD4AH9CAF+OMnKh9vDrvFIv23G3Yx3tZsCBYhYmISU6koVj0Xkit//pppzW+yPjJ+4z8LACTAO4TkZ+IyD0isshjABtFZFxExicnJ2N4WiIKImwpYNDHu2ftmzbVPg8SxIFkg3ixCFx+uffXFi0CbrjB//ud34F7wbSvb/6CZ+qNm70S52FuAAYBzAC4ZO7zrQD+zO97uNhJ1D5hN6kEebzXImArt7Bb5f1utkXLkZHaRqJCwXweZvz1C66lkvneJHatBoEENwRNAJhQ1SfnPv8OgP8Qw3WJKAZhN6kEud9W3x1WXDPycrlWNrh7t7lvwwbzDmHdOmBmxjzXzAywbVvw8TsnJDreftt89DudMQ2RA7mq/j8AB0XknLm7LgfwfNTrElE8wp7F7fX4vr7Gxwet5GiXoSEzLhETwJvVqgddmHS/0LSzPVwYcZUfXghTftgH4BcAPqeqv7Y9noudRO1VqZgA9NprwNKlwNGjwPHjta+7FzPdj3/zzcbqjjQXKb00W7h0L0TaXowKBZPf9/v5RGqVNe2WaPmhqu5T1UFVvUBVr/UL4kTUfvWpgMWLG4M4MH+m6X68u0QvS0EcaJ4mcc/AbVvpd+0yQd/v52tXe7gwuEWfqMuE7R+ZZn10XNzB128rvd/P6xwxkHRruLAYyIm6jG1G2dPjHZyyOAMNw1YaaFuw9Pt5nZl6kHNi2omBnKiLVCpm27uXkye9+1baHp8HtkOr/HaueqVdROZfO0sLnwvSHgARtUelAnzuc/Pz3fWOHQM2bzZldk7eOelt8kkpFGoz8YEBkzLp7zcVLrt21X4+Z3YN1I6yBWqLvf399iqdrKSdYqlaCYtVK0Ttl8Wywbj09c1fwAVMmaVI49dsFSl+W+xtv7t2bMuvl/ShWUSUUU4aoVOD+KJFwM6d3ueTnzgxP8Db5q5+s+u0GkYExUBO1MHqW6VlycKF8V3LSRVFre22LfYCKTaMCIipFaIOltWZ+MgIcPfdwR5bKJggvXQpcOSIPS0CBP9ZbemVsKc8thtTK0RdKCuLcW733hv8sQsXArfcYhZgbfPOAwfsRxH09TXeVyya63mlYrJUiRIGAzlRTnmV0G3aZM4CF8neNvp6XguTNm+9ZWbvfrs3nfLAG26oBehCAfijPzL5c3dKZNs2eyomqy9+flh+SJRDmzYB27c3blC58UZzul83Uq2VTTpnoZ88acoM163zriyxlRXmcQMUZ+REGefVwKE+iDu6NYg7qtVwDTSyXokSBgM5UYbVV504x7J6BfFWufPHnciWKsl6JUoYDOREKfPbLu7VACHOvPfNN3tvP+8kfqmSrDWIaBUDOVGb1QfuZcvMtnlbI4SkF97uvju7C6JxyGuqJCwGcqI2cqdKqtX5Z5/U53WXLm3/GLNkZKSW+iiVmr97KJU6I1USFgM5URsF7RWZxxK4JDgVJ7Oz5hTG3bvnL1A6ikVg69bOSJWEFVsgF5GCiPxERP4urmsSdZqgAdrJ6x45ktxY8sBdcVK/QAnUasa7afbtJc468s0AXgDw3hivSdRR/I5EddTndYM8vpN5/ez1R82SEcuMXERWALgapgEzEVnYOtQ7+V/3zHJ01CyKdiuR7HThybK4/hP5CwB/CsB6/piIbBSRcREZn5ycjOlpifLFSQ2USrX7liwxuV13XtfpZJ9Wx/YsUM3n2SftFjmQi8gnABxW1af8HqeqO1R1UFUHly9fHvVpidrOr947rLffrv27Wm0sOaxUTFni+vXdnVZxcOG3uThm5OsAfEpE9gP4FoCPichYDNclygyvHZZBmu96Bf/Nm+1byZ3nqVaT+knyJ49nn7RbrOeRi8hlAP6Hqn7C73E8j5zyxnaud6kELF5c6+04OtqYGtm4sTFo21qSkTcRU3LIxU2D55ETRWB7e1+t2mfpXjXj3RrEFy2qlQwGJWLODWcQby7WQK6qjzSbjRPlUdC39/W7MpnbrXnrLWB62v8x7uqd3bvNueHUHGfk1DVaWaysb1wc9HApJ4Azt9vIL+9fKpkGEFNT3bcrMw4M5NQVWlmsdDcuDrqc5ATwoaFoY+4mR44Ajz2W9ijyi4GcuoJXvrpZf8ag56K4OQH8gQfCf2+3UjUnMXLzT2sYyKkr2PLV9fe7+122WsN9//3Z7V6fdRs2MJi3goGcuoLtOFjn/k2bzIzQ6fcYhVPJQuGpBqvPp0YM5EQw2+YpG5qlvGg+BnLqCrbjYJ3745iJU3xYuhkOAzl1vErFfoKgU2HinGtN2cDSzXAYyKmjOSWEXjPuYtFUmAwMcEaeJd3SZzNODOTU0WwlhIUCcMMNwK5dXJhMW7f22YwTAzl1BNuuTVuudXbW1Hm3UidOrXPvju3mPptxYiCnTGl1G71t16Yt19rfzwW1disWzSFY7tk3EN85711LVdt+W7NmjRK5jY2pFouqJhybW7Fo7vdTLjd+j3Mrl1VHRlRFGu9fsGD+fbzFeysUGj+Wy95/x1b/5t0KwLh6xNRYzyMPiueRkxfbbshy2bzltunpsZ+DUiwyfRKVSPBzZuq/J0iLulb/5t2K55FT5gXZRu/Flj4pFBjE46Bqji4II2j5YKt/c2rEQE6Z4ZfP9uPVmb5YZElhnGZmgj82TPlgq39zasRATplhC8jNgoLTmd69iBa2Iw21rlBorXyw1b85NQr5hmk+EVkJ4JsAfhuAAtihqlujXpe6j/M//5Yt3j0wm32v1+M2bAif36XwZmeD5cTdovzNqSbyYqeInAHgDFV9WkSWAHgKwLWq+rzte7jYSe0StKsPRcPFyfZIbLFTVV9X1afn/n0UwAsAzox6XaKw6s8TLxRMd3tKHlMh6Ys1Ry4iAwBWA3jS42sbRWRcRMYnJyfjfFrqIrYNQ+7zxGdnTcNfSobzTodb6rMhtjpyEVkM4FEAo6r6Xb/HMrVCrXB2cLpLCksl4Ne/bi1HS8319AB//MfmSAPmsdNlS61EXuycu3gvgL8BUGkWxImCqlQaF8Gmp73rwv26s1Mjr809zn09Pd4vhqefDmzb1p7xUWsip1ZERADcC+AFVf1y9CEReZ+fwoAdTU8PsHs3MDbWWKrp3Gd7R2NrykHZEUeOfB2ADQA+JiL75m5DMVyXulirHezJmwjwzW/WyjTrTxsEzIumDTfnZF/k1Iqq/gsAFnlRrLhFO16qwObNwGOPzc91+71osiIlH2LJkRPFyWnNxi328apWTWWPwznu1++dDytS8oFb9ClVlYqp9xap3davZxBvl2PH7JumymUG8bxgIKfE2Wq/KxXg+utZ7502rwpkplTyhakVSpS79tt5Ow+Y3Cxrv7OnVDLt1zgbzw/OyKklQVuyeS2kHTtWqw+n7Fm8mEE8bzgjp9D8ZtnuAGAL1gcOmPNQmAvPHq+OPZRtnJFTaH6zbDe/GuSgQbxQCD42ik6EDZDzhoGcQgvTnsurcUBYnLW3l6r3izJlFwM5hRamPVd99x7KD65f5AsDOYUWpD1X/WLoli3A0BCbPGRB/fGzIyP2tBW35ecLAzmFZuuR6Sx0eh14tX07W66lpT54795t/g7795sTDXftYs/MThDbeeRh8DzyzjYwwMqHNPX2Au99rzm1MMjZ4e7jgnnWeHYl1uqNukPQunGA+dU0lUrAffcBU1O10w2bBWX3aYgM4vnDQE5NeaVKNm60B3NbfpU58uSImJz31BQDcTdiIKembHXj69fXZuf1M/bpaaCvb/51mCOPl7s5BLv4dC/u7KSm/FIlBw4AN91kgvSJE+a+atXkaUsldvVJSrlcawpBxBl5BwuT1/bTrBTt+PFaEHc4nzOdEj9WlZAbA3mHCpvX9jM6ambYYVWrTKfEpVDwLvUkAmIK5CJylYj8XEReEZHb4rgmRRPmPJQgOLNOT7Fo6r1ZVUI2kQO5iBQA/BWA3wdwHoA/FJHzol6XwnGnUWx13K2UBm7ZYtIn1H4inIFTc3HMyC8G8Iqq/kJVjwP4FoBrYrguBeSVRrHNoMNsvXZeHLi5Jz2qDOLUXByB/EwAB+s+n5i7r4GIbBSRcREZn5ycjOFpyeGVRlGdH8x7e01pYJDFz/oXByLKtrYtdqrqDlUdVNXB5cuXt+tpu4ItXaJaqzUulcxHZwHSKRtctsw7sG/e7N9dndqjVEp7BJQHcQTyQwBW1n2+Yu4+ahNbusSpNZ6dNe273Hnu48cbA/v69SbYL17M+u8s6OszvTOJmokjkP8rgN8RkbNEpA/AHwD42xiuSwEFOVY2TIqEXe3TVy4DO3cyP07BRN7ZqaozInIrgAcBFADsVNXnIo+MAnP+Z7edYFepmJk2a7rzQYS7NimcWHLkqvqAqp6tqv9OVbnnLCZhdmb6nWC3ZQuDeJ6wqQOFxbNWMipMp/pmeKxsfnD7PbWCW/QzKs6dmZzhtVepNH/NwlbXXyrZOy0RBcVAnlFhOtX7qVRM7bif3l5TqULxOHJkfiu8W27xXpDeupVNHSg6BvKMCtOp3sZJz/iVEpbLpqPM0aMmj25rxkvB9ffPX7PYts2/zylRFAzkGRWkpBDwXxD1Ss/UczabOPXjIsDJk3GMvnv55bibtVSL69hh6kKq2vbbmjVrlJobG1Mtl1VFzMexsflfLxZVzVza3IpF1ZER8/j6+3lL7lYomI9ef6Mwf2uvv2Wr16POBGBcPWKqmK+11+DgoI6Pj7f9eTsND7RKV7EYX3rE9rdkJyCqJyJPqeqg+36mVnKMZYXpiTvHHdfiNnUn1pHnWH8/Z+RpSGKWbPtbsnSUguCMPIfqzwln5572S2LDTtDFbSIvDOQ54z4nXOvOHS+XWQ+etEWLkikZHB5meSK1joE8Z2xNJJy3+9u3m+NP6y1YwJm7TaEAjIyY318QSZ7R3qw8kciGgbzNotYKN1sUGx42x586gamnB5iZ4aFZXgoF87tZt6757lcHc9aURVzsbKM4DsIKsijmXKv+ubpJT4+Z1TazceP8v4kf5qwpqzgjb6M4DsLyWhQDgKmpxrZt3dyqbUHA6cm6dfbdr85RBc5H5qwpyzgjb6M4aoWdQLJ5c+MZKm+9Vevs0+0lie6WdjZOIw4vs7NMR1F+cEbeRnEchOX4zW+ijYXMC97Spd5fYy6c8iRSIBeRu0TkRRH5qYjsEZH3xTWwThRHrfCmTcCGDTzcKi5vvjm/yoe5cMqbqDPyhwCsUtULALwE4PboQ+pcUWuFKxVTXsi3/PE5cQJYsoT125RvkQK5qu5V1Zm5T58AsCL6kDqbUyu8e7f5fMOG4GWI7L3ZOufIXi9HjrB+m/Itzhz5TQD+wfZFEdkoIuMiMj45ORnj0+ZP/e5M1VoZojuYu2vOu30Rs1VOJx7bph9bnpwoL5oGchF5WESe9bhdU/eYLQBmAFjnlaq6Q1UHVXVw+fLl8Yw+Q8Js9AlShugV7Lk7066318y6RcxH59/1qZLRUfM4t6NH2cSBcs7rkPIwNwA3AngcQDHo93RaY4kwTQHGxuwNCkRqj7M1hhBJv5FC1m5hGjqUSvZrEGUdkmgsISJXAfgygN9T1cD5kk5rLBG0KUCzXYT1j+/psefDCwVWrQBmxh1kB2c92++1lWsRtVtSjSX+EsASAA+JyD4R2R7xerkUdKOPXw9Nd8mbrY65XAZ27ZpfMteNWqn1jrOWnygrolat/HtVXamqF87dbolrYHkSNDj47eB0l7wNDc3Piff1mcOd1q8Pvnsxz0RqW+S9DA2FP4SM535TR/LKtyR969YcuS3v7c7Pel0PUO3pST8fnfStfp3A+V04zY3dt1KptYbFzZpaE2UV2Hw5WZVK7eyO/n4zw3PXI9ty5IsXA+95j6ln7u83Z6gEPVa103i1UfNbLwh6DaJOYMuR89CsmAwP2zeS1Af5pUtNysA54AowQdsJ3N1cK25LcYTtTcqGxdRteGhWwtz14NVqMsfLOp1uvI64zQO/rfG2vLZttyYXLqnbMJBHEGShzdaaLU6Fgqlk2bYNWLs23mu3g5MKsb2jsZ1Rs3UrFy6JAHCxs1VBFzjbsYGnp0d1ZMS+KJjlm9fvLMxiJBcuqZuAi53xCroJqBvPSCmVGpteNHuss8jrzKTdC8LFIk8kJAKS2xDUtYJuAhoaCna9RYuijScrREwVTlDVqpmbOweHebWoC9sOj6jbMJC3KMgmoEoFuPfeYNdbtswEtLEx/yNXs66/v/WqkWPH7DN5VqIQ2TGQtyjIDsEtW4LvwHRON7zhBhPMyuX8BXTn529WNdLTwn91rEQhsmMgb1GQbj+tzCKdw7AOHPBuQ5Zl9cfF+pVB+h1OVSqxEoUoLAbyCJxuP7bOMlFnkSdO1M7Zzrr6M1GcF7mwRExJYZR2eETdiIG8BUEPahod9Z5RFwq1xgfN1O8AzbKTJxu7HA0P2zvyeM26RYBbbqntkGXrNaLgGMhDCtqmDTABaOfOxhl1qWQ270xNmb6dfqf7OYKW8qXNXV1iW0fwmnXv3m02NBFReKwjDylo/XgzzZpMZFmxaB+3u0FDkMPEiCgY1pHHJGj9eDN+TSayrFSqzaa9uNcFmCYhSh4DeUhxdZiJoy66WAy3+SasRYsamxiPjZmUEOB9zC6rS4jSEUsgF5E/EREVkWVxXC/L4uowE7WixZkZv+c90a7jpVg0QXt62gTu+tm0kxJy5+2d8XDGTdR+kQO5iKwEcCWArth7F6R+3M2rysVW0RLU4sXmOY8caf0aXnp6aouWQU9zrB8PEbVf5MVOEfkOgD8D8D0Ag6o61ex78rzYGUalYs4Occ9ei0Wzg/Oee0yteCucRcUkD+XyOqyKXeiJ0pPIYqeIXAPgkKo+E+CxG0VkXETGJycnozxtpthqym0pCMDMaO++u/UgDtRSM812UUbhdVgVu9ATZZDX2bb1NwAPA3jW43YNgCcB/Nbc4/YDWNbsetoh55Gr+p9Jbmu0nMQZ3vXPF+RM8lIp+HN5NUNupeExEUUHy3nkTWfkqnqFqq5y3wD8AsBZAJ4Rkf0AVgB4WkTeH/NrTWZ55YudWWxSp/UVCiYts2VL7V0AYBYjVYGZGfPRVh4oEm6DkVc5IbfQE2VLbBuC5oJ5V+XI/fLFYRsGh+HekOPk3B94oLbxZmjI7CCtf5yI93gBU3Vy9GjjaY1s6ECULdwQlAC/fHFSuetCwftdwPbtjccG7Nplgnv9zNnvNXvxYhPEnSMDONMmyo/YArmqDgSZjXcSv5pydwqiVApWbtjXVzubxX2oVrFYO+bWzR2kjx0zM/T6XZV+6Rbn3cPJk40/AxFlH2fkETTLF9dvT5+aMgdo2YKpY+dO81hVc5CU+9rNvr+eO0/v9cLjlW5hazWinPFaAU36FqVqpRO6ptsqWkql5t/rVTUi4n29ctn7++t/f0GrVYgofWi1aiVLwhwhm2W2XZ3VKrBkCbBpk/28c693AbfcEvzYAPchVkEPvyKiDPOK7knfWp2R22aQXjPPJMT5biBsLffISDJjY104UX7AMiPP1XnkaW4P9zo/PEp5nu1nsRExOfMkFiB5ZjhRPtjKD3MVyONq6tCKuJ+7lTNS2vFzElF2dUQdeVxHyLbCtlOz1U0/o6PBenYGGQMRdbdcBfI0t4fbFv9EWltsHR42i5RxjIGIuluuAjmQXusw2wxatfWa623bTAMHd3Pmyy/33gzE7jtE5CV3gTwtw8P2xckoKY/h4doGIFXz74cf9t4MxAVIIvLCQA77meJu7ay5ZtNiIgqq6wN5mE1GaS62EhHZdH0g9ztT3I1ncRNRFuWqjjwJ7EFJRHnREXXkSWAPSiLKu64P5Mx7E1HedX0gZ96biPJuQdoDyILhYQZuIsqvyDNyEfmiiLwoIs+JyP+OY1BERBRcpEAuIh8FcA2AD1dtbF0AAATeSURBVKnq+QD+PJZR5UDQTUREREmLmloZAfC/VPVdAFDVw9GHlH3us8mdTUQAUzRE1H5RUytnA/hPIvKkiDwqIhfZHigiG0VkXETGJycnQz9RlmbAYTYRERElremMXEQeBvB+jy9tmfv+pQAuBXARgPtF5APqsctIVXcA2AGYDUFhBpm1GbDtkCyeF05EaWg6I1fVK1R1lcftewAmAHx3rp3cjwHMAlgW9yCzNgPmJiIiypKoqZX/A+CjACAiZwPoAzAVdVBuWZsBcxMREWVJ1EC+E8AHRORZAN8CcINXWiWqrM2AuYmIiLIkUtWKqh4HsD6msViNjnp3sE9zBsxNRESUFbnYos8ZMBGRXW626HMGTETkLRczciIismMgJyLKOQZyIqKcYyAnIso5BnIiopxLpfmyiEwCOACznT/2naAJ4Vjjl5dxAhxrUjjWcMqqutx9ZyqB/NSTi4x7dYTOIo41fnkZJ8CxJoVjjQdTK0REOcdATkSUc2kH8h0pP38YHGv88jJOgGNNCscag1Rz5EREFF3aM3IiIoqIgZyIKOdSD+QicpeIvCgiPxWRPSLyvrTHZCMinxGR50RkVkQyV4YkIleJyM9F5BURuS3t8diIyE4ROTzXkCTTRGSliPxARJ6f+9tvTntMNiKyUER+LCLPzI31f6Y9Jj8iUhCRn4jI36U9Fj8isl9EfiYi+0RkPO3xeEk9kAN4CMAqVb0AwEsAbk95PH6eBfBfAfxz2gNxE5ECgL8C8PsAzgPwhyJyXrqjsvoGgKvSHkRAMwD+RFXPg2ky/oUM/17fBfAxVf0QgAsBXCUil6Y8Jj+bAbyQ9iAC+qiqXsg6cgtV3auqM3OfPgFgRZrj8aOqL6jqz9Meh8XFAF5R1V/MdW76FoBrUh6TJ1X9ZwBH0h5HEKr6uqo+PffvozCB58x0R+Vtrgn69NynvXO3TFYziMgKAFcDuCftsXSC1AO5y00A/iHtQeTUmQAO1n0+gYwGnLwSkQEAqwE8me5I7ObSFfsAHAbwkKpmdax/AeBPAcymPZAAFMBeEXlKRDamPRgvbekQJCIPA3i/x5e2qOr35h6zBeZtbKUdY7IJMlbqPiKyGMDfAPhvqvpm2uOxUdWTAC6cW2vaIyKrVDVTaxEi8gkAh1X1KRG5LO3xBPAfVfWQiPwbAA+JyItz7yozoy2BXFWv8Pu6iNwI4BMALteUC9ubjTXDDgFYWff5irn7KCIR6YUJ4hVV/W7a4wlCVX8jIj+AWYvIVCAHsA7Ap0RkCMBCAO8VkTFVTbyReytU9dDcx8MisgcmjZmpQJ56akVEroJ5i/UpVT2W9nhy7F8B/I6InCUifQD+AMDfpjym3BMRAXAvgBdU9ctpj8ePiCx3qr5E5DQA/xnAi+mOaj5VvV1VV6jqAMx/p/83q0FcRBaJyBLn3wCuRPZeGNMP5AD+EsASmLcs+0Rke9oDshGR/yIiEwDWAvh7EXkw7TE55haMbwXwIMyC3P2q+ly6o/ImIn8N4HEA54jIhIjcnPaYfKwDsAHAx+b++9w3N5PMojMA/EBEfgrzwv6Qqma6tC8HfhvAv4jIMwB+DODvVfUfUx7TPNyiT0SUc1mYkRMRUQQM5EREOcdATkSUcwzkREQ5x0BORJRzDORERDnHQE5ElHP/H8Ykd4P3kWclAAAAAElFTkSuQmCC\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nz4nFUT8m0wO"
      },
      "source": [
        "Our goal is to create a generator that can generate a distribution that matches this distribution."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1InC1BAQnC6l"
      },
      "source": [
        "### Generator network $G$\n",
        "\n",
        "Now we are ready to build our generator network G, to keep things simple, we make generator an one layer linear neural network."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7MPePSoJm_YW"
      },
      "source": [
        "model_G = nn.Sequential(nn.Linear(2, 2))"
      ],
      "execution_count": 10,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ibevU5pCnStH"
      },
      "source": [
        "def sample_G(model_G, num_samples):\n",
        "    Z = ndl.Tensor(np.random.normal(0, 1, (num_samples, 2)))\n",
        "    fake_X = model_G(Z)\n",
        "    return fake_X.numpy()"
      ],
      "execution_count": 11,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2cifleRqj-j2"
      },
      "source": [
        "fake_data_init = sample_G(model_G, 3200)"
      ],
      "execution_count": 12,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 283
        },
        "id": "n-ZhutLVkGgU",
        "outputId": "0baf6625-7c0b-4a81-df5f-c7d8c4217a37"
      },
      "source": [
        "plt.scatter(data[:,0], data[:,1], color=\"blue\", label=\"real data\")\n",
        "plt.scatter(fake_data_init[:,0], fake_data_init[:,1], color=\"red\", label=\"G(z) at init\")\n",
        "plt.legend()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.legend.Legend at 0x7fb60d4aed50>"
            ]
          },
          "metadata": {},
          "execution_count": 60
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXIAAAD4CAYAAADxeG0DAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3df5QcZZkv8O8znRkmPUkQOqBcwvTEoygk5IeJQIyyrPEKJijqwYUwGSJBc5gAN15XvEDOEf4Zr1w8xnCWBCMmZDOtHnTF1T3sGvm5gsBxkCArsIE100MAYTLZhPwgZDL93D/eqZnunqrqqu7qrqru7+ecOpmurq6u7pk8/fbzvu/ziqqCiIjiqynsCyAiosowkBMRxRwDORFRzDGQExHFHAM5EVHMTQrjSadPn64dHR1hPDURUWw988wze1X1lOL9oQTyjo4O9PX1hfHURESxJSJZu/1MrRARxRwDORFRzDGQExHFXCg5cjvDw8PYs2cPjh49Gval1KXW1lbMmDEDzc3NYV8KEQUsMoF8z549mDp1Kjo6OiAiYV9OXVFVDA0NYc+ePZg5c2bYl0NEAYtMauXo0aNIpVIM4lUgIkilUvy2Q+RDJgN0dABNTebfTCbsK3IWmRY5AAbxKuJ7S+RdJgOsXg0cOWJuZ7PmNgB0doZ3XU4i0yInIoqKdevGg7jlyBGzP4oYyAPU0dGBvXv3uh5z77334vrrr3c95tFHH8Xvf//7IC+NiHwYGPC3P2wM5DZUFblcLrTnZyAnCld7u7/9YYttIA+6I6K/vx8f+tCHcNVVV2H27Nl49dVXcccdd+CjH/0o5syZg1tvvXXs2M9//vNYsGABZs2ahc2bN5c899atW3HmmWfi3HPPxRNPPDG2/9e//jXOO+88zJ8/H5/61Kfw5ptvor+/H3fffTfWr1+PefPm4Xe/+53tcURUPT09QDJZuC+ZNPsjSVVrvi1YsECLvfDCCxP2OentVU0mVYHxLZk0+8u1e/duFRF98sknVVX1N7/5jX71q1/VXC6nIyMjumzZMn3sscdUVXVoaEhVVY8cOaKzZs3SvXv3qqpqOp3WwcHBgvO+/vrresYZZ+hbb72l7777rn7sYx/T6667TlVV9+3bp7lcTlVVf/jDH+rXv/51VVW99dZb9Y477hg7h9Nxfvl5j4kaXW+vajqtKmL+rSS+BAVAn9rE1EiNWvHKrSOikh7ldDqN888/HwCwY8cO7NixA/PnzwcAHDp0CC+//DIuuOAC3Hnnnbj//vsBAK+++ipefvllpFIp23M+/fTTuPDCC3HKKaZg2eWXX45du3YBMGPnL7/8crzxxhs4duyY4xhvr8cRUXA6O6M5QsVOLFMr1eqIaGtrG/tZVXHzzTdj586d2LlzJ1555RVcc801ePTRR/Hggw/iySefxHPPPYf58+eXPT77hhtuwPXXX4/nn38eP/jBDxzP4/U4ImpMsQzkteiIuOiii7BlyxYcOnQIAPDaa6/hrbfewoEDB3DSSSchmUzipZdewlNPPeV6nvPOOw+PPfYYhoaGMDw8jJ/97Gdj9x04cACnn346AGDbtm1j+6dOnYqDBw+WPI6I4qOaE4xiGchr0RHx6U9/GldeeSUWLVqEc845B5dddhkOHjyIiy++GMePH8dZZ52Fm266aSwV4+S0007DbbfdhkWLFmHx4sU466yzxu677bbb8KUvfQkLFizA9OnTx/Z/9rOfxf333z/W2el0HBHFgzXBKJs1vXrWBKOggrmY/HltLVy4UIsXlnjxxRcLglwpmYzJiQ8MmJZ4T0988llh8fseE1EwOjpM8C6WTgP9/d7PIyLPqOrC4v2BdHaKyP8G8BUACuB5AFeralUTuXHqiCCixlbtCUYVp1ZE5HQA/wvAQlWdDSAB4IpKz0tEVC+q3a8XVI58EoDJIjIJQBLA6wGdl4go9qrdr1dxIFfV1wB8F8AAgDcAHFDVHcXHichqEekTkb7BwcFKn5aIKDY6O4HNm01OXMT8u3lzcOnhIFIrJwG4FMBMAP8DQJuIrCg+TlU3q+pCVV1oTY4hIqqFKNQW7+w0HZu5nPk3yD6+IFIrnwKwW1UHVXUYwC8AfCyA8xIRVazaQ/+iIIhAPgDgfBFJilm9YAmAFwM4LxFRxeJWW7wcQeTInwbwcwB/hBl62ASgdEnACHrzzTdx5ZVX4v3vfz8WLFiARYsWjdVUAYBnn30W11xzjePjn3/+eXz5y1/29Zz9/f348Y9/bHvf66+/jssuu6zkOZYuXYr9+/dj//792Lhxo6/nJ4qyIFIicastXha7SlrV3iqtfqiqgZcmy+Vyev755+umTZvG9vX39+udd945dvuyyy7TnTt3up5nyZIlms1mPT/vI488osuWLfN/wTZ2796ts2bNcryf1Q8pToKqcppOF57D2tLpalx1dcGh+mE8A3kV6tg++OCDesEFFzje//bbb+uZZ545dvszn/mMzp07V+fOnavTpk3Te++9V1VVv//97+vtt98+4fG7d+/Wj3/84zp//nydP3++PvHEE6qqet555+m0adN07ty5+r3vfW/CY6zAvHXrVv3CF76gF110kX7gAx/QG2+8cew4q3zu5Zdfrq2trTp37lz9xje+MeEaGMgpToIKwNUoex2W+grkVfiI3bBhg37ta19zvP/hhx/WL37xixP29/X16TnnnKP79+9XVdXHH39cL7nkkgnHHT58WN955x1VVd21a5da74Fbi7w4kM+cOVP379+v77zzjra3t+vAwICqjgdytsipnojY/zcX8X+uKNYWL4dTII9lPfJaJL2uu+46PP7442hpacEf/vAHvPHGGygeNrl37150dXXhvvvuw4knnggAOPXUU/H66xPnQw0PD+P666/Hzp07kUgkxmqS+7FkyZKx5zn77LORzWZxxhlnlPHqiKKvvd2+Pkk5syHrvaRHLKsfVmO+66xZs/DHP/5x7PZdd92Fhx56CNbkpcmTJxfUAR8ZGcEVV1yBb33rW5g9e/bY/qNHj2Ly5MkTzr9+/Xq8973vxXPPPYe+vj4cO3bM9zWecMIJYz8nEgkcP37c9zmI4iJ2y62FKJ6BvAq/4U9+8pM4evQoNm3aNLbvSN6YpbPOOguvvPLK2O2bbroJc+bMwRVXFJaV2bVrV0Fgtxw4cACnnXYampqasH37doyMjACYWHu8EkGeiyhs1Z4NWU/iGcir8BsWEfzyl7/EY489hpkzZ+Lcc8/FypUrcfvttwMAPvzhD+PAgQNjgfK73/0uduzYgXnz5mHevHn41a9+BQB45JFHsGzZsgnnX7NmDbZt24a5c+fipZdeGluNaM6cOUgkEpg7dy7Wr19f9vUDQCqVwuLFizF79mzceOONFZ2LqFbchhhWczZkPYltPfIwrF+/HlOnTsVXvvIV2/vfffdd/M3f/A0ef/xxTJoUve6HOLzH1FisWZf5E3aSSba8nTjVI49nizwk3d3dBXnqYgMDA/jOd74TySBOFEW1mHUZhTor1RapQB7GtwM/Wltb0dXV5Xj/Bz/4QVx44YW1uyAfov7eUmOq1gA0K3iLAF1d/uqsxDHwRyaQt7a2YmhoiAGnClQVQ0NDaG1tDftSiAo4DTRTLT+I5hfJss6Vz63Fb1dgq6vLfCCUup4wPwAikyMfHh7Gnj17Cob4UXBaW1sxY8YMNDc3h30pRGPscuT5ysmXO62PmU/EdKD6fazT9dQq1++UI4/MzE4iijY/syPtjnV6vLXfbhYnoJpI+JuJ6TQj1Msk8HIfW6t6Loj6FH0iii4/9Ursjm1pUW1udn+8WxD1UxvF7UPBmuLvdK5Sj3UqERBkOQE3ToE8MjlyIoouP6NL7I49dgwYHnZ+fCZjcstO/Ixk6ekBWlqc71d1TnfYzTUsZpfXr/biyqUwkBNRSX5Gl/gZcTIwMJ5fHp3s7KhU3tvS2QlMnep8fyLh3BFpzTVMpezvd5pAHnY5AQZyIirJT4vTTyu0vR1Yu9a5s7OY15Eg+/Y53zcyUnoI4jvvTNyXSjl3XoZdTiCQQC4i7xGRn4vISyLyoogsCuK8RFRbTkPo/LQ4vbZCk0lg6VJgaMj79a1da3/N06ebACpifj75ZPfzuKVq7FJDADBlintgDrWcgF3i3O8GYBuAr4z+3ALgPW7Hs7OTKHpKdWj6GbXS1ubeYZhIqHZ3e+tcLN6Kr7mlZeIxTU32+710RNaq47IcqFZnp4icCOACAD8a/WA4pqr7Kz0vEdWWU4fmypXAmjXm/mzWtNazWXPbLj2RyUzs2Cw2MgLcfbf3vHfx+fOv2a4idC5n8uTptPN5/HZQ1qrjshxBpFZmAhgEsFVEnhWRe0SkLYDzElENOXVSjowAmzaNB12rU9JpurtTcC2mZc5FzH9Ot47VfftMiqO3119HZNgdl+UIIpBPAvARAJtUdT6AwwBuKj5IRFaLSJ+I9FmLNRBRdJTT4rTLNVd7dfr853S7Zus+vx2RYXdclqPiKfoi8j4AT6lqx+jtTwC4SVUnFuUeZTdFn4jCVWq6vJt02rTQE4nSwwiDYE2xz2SAVasmfgNobga2bo128C1H1crYqupfAbwqIh8a3bUEwAuVnpeIastqiSYS/h9bnHYp1tzsPknHjgjQ5pCkzW9tb9lSOO47larPIO4mqHHkNwDIiMifAMwD8O2AzktENRZ0kUwR0/k5dar52StV4PDhifubmyfmq6dMGU+DbNhgH8Tzh1ZOn262OJWqdROZ6odEFK5SqZVapU1KmTLFtLoHBsx48YMHC1Mr+VUHM5nx0TYizh2scVmViCsEEdEYu4k/ThNhLCMj7vVQauXQofF64UNDE/PjVmdoqbrkdo+JK7bIiRqMU+1sL52cbq3aKBExeXS/49Sj/trYIiciAM4Tf7x0ckY90Fna2/0PgxSJb66cgZyowbhN/ClVwrXWyknltLSYzlC/4+JV45teYSAnajBuAW7yZOcSrmHI5fwPh7S+NXipLV6s2pOZqoWBnKiO2XVqugW4oSEztd3vmO9K9Paa4Os0NNHvSJnhYdOyzp+h6VWU66m4YSAnqlN2K8KvXm3uc1s8QdVbrZSgWEP+ggyi2az54OrqMre9fMuIej0VNwzkRHXKbXm2J55wX3yhVvJz4D09ZrKPH04BWqTwA+zgwYnnbmkxj49LPRU3DOREdcop35vNmhKyURiBksuZlvOaNWbRiFLlb/OlUmYWZ3GayG6I5LFjwLRphYWwtmwB9u4NaSGIgDGQE0Wc06o9pY5xS1VEIYhbsllTJtfPSkEi41PxiysVOr026/zbt8c/cBfjhCCiCLObvGO1ONPp8Zyu3QSflSuBbdvKq2YYZSLAtdcCGzfa39/R4T4RKC7T8e04TQhiICeKMC9BqanJTFsvZuWP/bR0oyiVMvVVBgbMt4yeHvcg7KUcbzptWuVx4xTIJ4VxMUTkTakp5m7BKu4BHChMoXhlHWsVy7IT1/HiTpgjJ4qoNWvCvoLwqZaXArFWtHcaQx7X8eJOGMiJIiiTMSNLGp2fyTx24rj+ZjkYyIlCUGokyrp10RpZEpalSyt7fBzX3ywHOzuJasypjGx+gGlqYiAH6nftzXJVvYytiCRE5FkR+ZegzklUj9xmXFrqLYdbLqtuCrkLMrWyFsCLAZ6PKNac0idOIyby95dTua9e1dsIk2oIJJCLyAwAywDcE8T5iKLGy+zK4uPtCla5zbg8+eTxn63cbhSWVquFtjbn+/jtpLSg/ky+D+CbAHJOB4jIahHpE5G+wcHBgJ6WqPrcgrITt/RJT499mdj//u/Cld0BUwekEfzgB/bvSXNz/Y0wqYaKOztF5BIAS1V1jYhcCOAbqnqJ22PY2Ulx4jS70m12oFtnZTrtbS1Jr+to1oPeXvPv2rXjE5msoljs6BxXtSn6IvJ/AXQBOA6gFcA0AL9Q1RVOj2EgpzhxCsoiE1vM1mr0ToE6LosX11pcp8zXWtVGrajqzao6Q1U7AFwB4GG3IE4UdVY+XASYNMk58BbnbvNTME4YxO35Xe2eCjVIVwqRN8XB2GmZMbvZgXZ5cYvTMmZkxHkF+ygINJCr6qOl8uNEUeYWjC1OswPdhsmxJe4uzivYRwGrHxLlKTVmWcQ5l9vezhRBJThevHxMrRDlKTVm2e1+TuKpDMeLl4+BnCiPWzBOJk0Rp+KJQVbnaFcXMHly40ziCVI9ViSsJaZWiPIUL0qQSJgOz3TaBPEf/cgs5AuY+1cUjc+qh8UcaqWpyQzftJas43jx8jGQExXp7LQPKtOnjwdxKl8iYdYSZeAODr8EEnnE1nYwcjkG8aAxkBNRTbFTM3gM5EQ27KodWqvSU/lE2KlZDQzkREUyGWDVqsJqh6tWAX/3d6YaH5VHBLj2WqZVqoGBnGiU1QpfsWJip+axY8B995llxypdENiyHBnsRgdG0ITd6MByxGOOeiLh7TgR8y3GWitz+3Zg48bqXluj4qgVItivo1ksyM7O5chgC65GK4YBAB3IYguuBgD8BNFtsnotrVu8BilVF1vkRPBWYwUAHliRwaPZylvRG7B2LIhbWjGMDVhb1vlqQQRYtKj0hKdEgkG81hjIieCtzsdyZLAZq9GBLJqg6EAWP8TqsoL5dNg37532R4Eq8PDD7qsWJZPuY8T9LplH3jCQU8NwCyJehsR9G+vQhsJmexuO4NtonLJ9pao4urXEy1kyj7xhIKeGUCqIeCl41Q77ZrvTfjd74TyWMS6dnsXSafd0its6plQZBnJqCE5BZO1oSrqzE1i50uSBnUaTDMC+2e60381abIBd41aAWLbwvRS9ckpfsXxt5RjIqSE4BYuhofFW+QMPAFdoBj90yIPfgh4cRmGz/TCSuAX+Z7j8BJ22gRwor4UfplTKW+emU/qKMz0rV3EgF5EzROQREXlBRP4sItHtdqeG5RYsVqwwLfFs1j0P/hN04qvYjH6kkYOgH2l8FZsnDBf0Oj58APYD0stp4YchnQZ6e4G9e72NULFLX7F8bUBUtaINwGkAPjL681QAuwCc7faYBQsWKFEt9faqmuy4+zbicMcIZMLu5ejV3UjrCER3I63L0avL0auHkCw48BCSuhy9to/3emzUNpHx9zWdNrfTaXO71O/Bz/FUCECf2sVhu52VbAD+GcD/dDuGgZzCkEo5B6bl6NW3kNKcwwHDSJQM2COjAd/u8buRdnze4g+DsIO01y2VUm1uLtyXTDI4V1NNAjmADgADAKbZ3LcaQB+Avvb29tq8ampcNk2/3l4TaLy0jPO34uA+AnEM+G7niFOQrmRLp0P+3dcxp0Au5r7KicgUAI8B6FHVX7gdu3DhQu3r6wvkeYkmsJlvn4MZETJ+uwmCHAaQRhsO4ZQaTMQ5jKRtTr3eiLhPGqLyicgzqrqweH8go1ZEpBnAPwHIlAriRNWUyQB7Vk4ca9gEE8itLYEcmmBqnNRqNmWjTB7iKJTaC2LUigD4EYAXVfV7lV8SkT/WjM0rJYPFKzpw+kjW1+Ol9CGBidvQQr84CiUcQbTIFwPoAvBJEdk5ui0N4LxEzkajt4rg8hWT8JesIIMV6EC2poHZr7gMLfSqpaWwVC2LZYWj4kCuqo+rqqjqHFWdN7o9EMTFUQNzK4ySt/KDAJiEkbHUSdjcepzKnTxUa4nEeGDu7jb/Ot3essWMI8/lgP5+BvGwsB45Rc7jazL4yN2rkdTRPLdVGAUwkWLt2tgtZ38cidh0dOZy7KyMGwZyipRMBvjE3evGg7jlyBFTDOWJJyK9nP1BtCEBLZgdGrfRKuysjB/WWqFIWbcOmKEOHYIjI8CmTbW9IJ/eRaunafxhE4c8FDsr44mBnKJhNCf+l2zT6MDAeEphH36CTsxEPxLIYSb6IxfEATN1pxg7K+Mrvv9jKD7cOi7XrDH7V6wAsqbi4CSMuHYaRllcR6Wk0+ysjDPmyKm6imdZZrNAV5dZjv7hh+2bhojGCJR8CpP/PgHDOAH2Ha1xGZVihzXB440tcgpWJgNMn26SsCLAVVdNXNFBFXjoIccgHiUK4G20oRO9OBGHcDW2jOW/B5HCIFKRzoUXSzksTMQOznhji5yCk8kAV18NDOetDh/zcWxZpDET/WO3f4LOyAdrJ6kUsGHDhDI07OCsA2yRU3DWrSsM4jGXg8Q2VVKsudkE8c5O06GZP6mHHZzxF1j1Qz9Y/bBOOY1pi6kcgERsu13HWS1xBuv4q2r1Q6ozbqNM3I6vM05LscVBd/d4hXCvS7FRfDGQUyFrlEk2a6KANT2+uNZJR8d4h+bo0MF6EocRKFaxqrY285kLmDop3d3Axo3+P48pvphaoUIdHfZBWSQWo0wqpQD2IoW12BD5Tk1r7Lcdm7U1kEwyHx53TK2QN04DiusgiCuAQaTwLloK9ucgyAHoRxqd6MWp2Bv5IA64j/1eN3FtDRw5YvZT/WEgp0JtbWFfQVUMIzEWpPPHgvcjjRXYjgQ0stPpnbiN/XYK8pz4U58YyMmwJvIcOhT2lQRGMT6hZyW2jQXpONRCKUXEfey3U5DnxJ/6FNSanReLyH+KyCsiclMQ56QqyO/9mj59fAamVeskwuVhvbCCttXS7kQvmqA4EYdiGaydiADXXuue6+7pMTnxfJz4U7+CWLMzAeAuAJ8BcDaA5SJydqXnpYDkjzDp6hofjTI0NB646yD/DZhOyhNxKNYt7VLSaWD7dmDxYvcRKZz401iCaJGfC+AVVf2Lqh4D8FMAlwZwXqpU/lBCoG4CtpOpeBvLUb9j7ETGR6mUGiEKmKDd389l2BpBEIH8dACv5t3eM7qPwmK1wlesmDh0oY61YhjfRv0Oy7Dy2xyRQsVq1tkpIqtFpE9E+gYHB2v1tI3DLoVSJ6xOS2uY4CBSjhPn21G/wzKs/DZHpFCxIAL5awDOyLs9Y3RfAVXdrKoLVXXhKaecEsDT0pg1awqDd52lULJIowmKBHJIQHEq9iLrMH0+bgs7FHdIOkmlxlMjHJFCxYII5H8A8EERmSkiLQCuAPCrAM5L+ZzmW8+aZdaxrLPgbXGaKn8LenAYSU/HRtmiRaYj0k0yaYpeWTgihSZQ1Yo3AEsB7ALwXwDWlTp+wYIFSj709qomk1YNJLM1N6smEoX76mDLAfoWUjoC0d1I63L0Oh6+HL26G2lPx0Z1E1Ht7p7467W2dNr8+u3+JNJp83inY6j+AOhTm5jKWitRl8mYtEkIv6cw1EvpWD/SadOaXrfO5Lnb281tjjKhYk61VrhCUFRlMsDatbGfpONXnEvHlmtgwARtBm4qF6foR401Vb4OZlo6UZgZmEfRXLA/jjnuILCTkirFQB4l1gSeOgzg1vBBa+r8iTiEVdhaULwqDosXB42dlBQE5sijxKkWeB3oL1rEuFFNmQKccAKwbx9z4eQfc+RRl8nUbRA/iuaGTJkUEwEOHgz7KqgeMbVSa1YO3FomralpfLm0OqF52yBSWIWtDZcyscNcOFULA3m1FZeOLe7ErLNhhYeRHCsf2zQ6C5NBHJg0yaRRuI4mVQNTK9VUvHBiHXVi5n/85NAEQQ4DSOMW9DBwFxEB7r3X/Jz/52BVLQSYJ6fKMJBXk12ZujqgAO5CN27AxrAvJfLyFzzu6HCuWshATpVgaqVa6rjzMos0g7gHxYs5sGohVQsDeTVYKZU61KiTdoq1tADd3e7HFC/mwKqFVC0M5OWy67XKZMxA4Tpa0OFdtGAQqYaetFNMBNiyxSy35qa4M5NVC6lq7CppVXuLdfXD3l7VVCr8snnV2iZNUk15qz7YqJuI+VNIp0sfm0wWViZk1UKqBFj9MADFo1BiSAFI3u2jaMZBTEMK+3Ak1Y4pG8xUw6YmE4poonTapE28vkfW8USVcprZydSKH2vXxjqIH0YSd6G7oL7JKmzFqdiLBHJ47zv9yIymTZi3NVpaCm/np0K8vkfszKRqYyD3KpOJ5zjwRAIQwZ6EyW/fgI2YiX4kkMNM9Bfku/MX8O3pmRjEGk0qZXLh6bTJixePQrHLedvhhyJVW+MGcrcpdnb3rV0bznVWIpkEtm0Dcjl8e3U/fiqlOymzWfOSARPEmurwL6StrfQxLS1mebXOTpMWyeUmjkLp7DSB3Qr0qRTQXFiZl52ZVBt2iXOvG4A7ALwE4E8A7gfwHi+PC72z027pNKtXyu6+OG55PWnlvKRk0ixBFvbLCHprazPvR6njurvL/9NiZyZVC6rR2SkinwbwsKoeF5HbRz8Y/k+px4Xe2elULtZaBTduE3mmTAHuvttxemC51XFF6rPDM5ks3dWRSJgvM5xxSVFSlc5OVd2hqsdHbz4FYEYl56sZp96nbDbaQTydBnp7C5O2vb2mNqpNxLEKLZb7kuoxiAMmiIu4HzMyYgYosagVxUGQGdBVAP7V6U4RWS0ifSLSNzg4GODTunDKg8ex98lKtrolbfNkMsCqVfHsn60F1Yn57GL5nb9EUVYytSIiDwJ4n81d61T1n0ePWQdgIYAvqodcTU1SK3Zjvq0KRgBw1VUmGMaBtcy6j+/5dbzYUCBEgGuvBe67r/SHXb1+M6H4cUqtTEia+90AfBnAkwCSXh8TeGenXQ+T07S7dNo8pqkp/J63UtukSWX3lomEf/lR36w/BbdjRNhhSdEBh87OilIrInIxgG8C+JyqhjNTxmp5Z7Pm/55V5NmpOWrlx6PUGk+lTK67OP99771l97adfLL7/S0t9Tm00A/rT8Hq47ajyvQKRV+l/5X/AcBUAL8VkZ0icncA1+SPXc3vI0fMsAM7Vn7c6f5qsXrXUimz5XdW7t1rAvZo/juzPYcO9KOpq7OsVWQyGfe1IVMpYOrUaH2WhcH6Uyg1sYczMyny7Jrp1d4CTa34ySHkVzCq5SDpVMrzy3Eb4u50vNesUn6aoNFTL3bFrBIJ+2OtFAxR2OCQWvEdhIPYAg3kXkrQWcG0OBouWVKbqGGVy6vg5dgFE6egX+py0un6LuBo/bpbWib+GqzXb/fB6PdDlKjWnAJ5fLKkTkMJe3pKjyOz5OebMxng3/89uOtLJEzOwo6P4Y5+VpHxm1WyZLMm9eL1bYubZNJMry+uk7J9uwnP1kreO2IAAArFSURBVKjN4j8poHDKfXFtFaLIsovu1d58t8hLTakvbno5bfn8NkndchFu0/t9Nun8tMgrTY+0tYXfcq7W5pQOyU89Fb9/bH1T1CHWqRW36OY1tVIcyP1EBdXCZHQqZTa7ghoVFtvw81ng9rbUe+qk1GaXzfJSc4b5cIoyp0Aej4UlnCr4WyNBvLyGVMqMDil+rN/H1UAmY9ImAwMmK+M0F8huzpM10QUANm2qzfVGkd1iDl4mSYlwNA9FV7wXlnBbtdZL/jmRMEnTfE757AjwOAsfnZ3AypWFn0mqptjTfffV4kqjqaUFOHRoYneKl2GETU32lY2Joiwegdxt1Vov1f3tZr5s2OBt5YR9+7xfZwgeeGDiF5IjR+q3xkqpL1KplHk/hobMv9b8sEzG22f+yMjExxFFnl2+pdpbWcMP3XLP+ff5GQzsNug6JknTuIwHTybd8/ZTpozntp1y3laN8OL1r/NHlrr1G9jlyK3n4xhyigPEurPTD7dI4CYmg4iLP8+cgmMqFa31MVIpE4jL+eBJJMZ/DaV+TaV+/U7tgXL/bIhqqXECuZ/xe8UivryLXRBrbp44+jJ/NGT+QBur1RvWVu4HS34wLfXrLffXX8mfDVGtOAXyeOTI/XDLp5fitZcxJHYTgIaHzZZv8mTzb/7L2bBhvH/XmjCUTgNnn13VSy5QalUeJyefPD5xp1QttHJ//ZX82RCFzi66V3ur+pqdEW9Z+5H/Uvy2fru73Se/BF2hIH94fVDntPvGUarlXO6vv47+bKhOoWFSKxHmN1BUug50LTtCi+uYOHUelvPh4OVDi0GXGoFTIK+/1EpEOZVNdxveZpdK8UO1/MeW+1zW6xoZqfyc6bT76E/WQyEyGMhrxKnAlduiBW4TWERqX1Ldq0o+fCxWftpp7Hc6HdmuDKKaYyCvET9VDS2lgti2baXnQkWZNbknnQa6u+2rDrITkqg0BvIacasy4KRUEOvsHC+7CpTXQm9r8/+4pqbKS+AmEoVlZTdutB8wlP8amUohshdIIBeRvxcRFZHpQZyvHpXTsvQSxKwhhqrA8eNm5Tg/Dh8GZswwP3upIyYC/OM/Alu3jn+AeK0/li+X8x6MIz4qlCh0FVc/FJEzANwD4MMAFqhqyVKBvqsf1gmvVQ0r4aXCXz4R/52ixcdbr8vP89pVJyQid9WsfrgewDcB1HCMRDwF3bK0WzTJ70LBfoO43Yrz1uvy2jJnjpsoWBUFchG5FMBrqvqch2NXi0ifiPQNDg5W8rQNxWmFO6fhjCefXL1rKRWAnfL9qRRz3ETVNKnUASLyIID32dy1DsAtAD7t5YlUdTOAzYBJrfi4xoZVvHCEFawB5+GMkyeXly6x09wMTJtmxnJ7SQX19Exc6MJaP5OBm6h6ys6Ri8g5AB4CYP23nQHgdQDnqupf3R7bqDlyv/zmu4Hggng6XV4Ovxb9AESNyilHHthSbyLSD2AhOzvtlRPgnFa4c5NIVD6rksudEUVTvJd6i7lS0/Od8uBeVrQpFsTU+HKel4jCUzJH7pWqdgR1rnpTanq+Ux7cLudcbRxRQhQ/gQVycuY2Pd8tyFvjrN3GaKdSwDvveA/2TU3OaRMRjighiiOmVmrAbXp+qRos1hjt3l77maGAvxb7SSeZuibFmpvNlHkGcaL4YSCvAafp+UuXmhayHSv4W/nzri4ztDCVKhyP7Vbm1c7QELB4sflgyB/bvXUrgzhRXDGQV8Cpk7KYXc2UlSuBe+6x75y08tTFnaRDQyaNsn37+MzQcjomrRy80yxTr6+LiCLCbrWJam/1sEJQqdXcS3Fa+UZk/BxeFgR2uo7ubvfVhZwWFbY7n4g5HxGFCw4rBAU2jtyPehhH7jRZx2sxKLe6JNavxGkcefE4b6cx6pkMsGKF8/PbdXo6vS4R5tCJwlb1CUF+1EMg9xpknXgJ5JV+WJRzDrdJSKxYSBQuTggKWDkLReRLpUrvD2J1HL/ncLt+v5UViag2GMjLVGmQ3bABaGkp3NfSYvZbglgdx+85enqcvy1wxidRRNklzqu91UNnp6rpGEynTWdgOu29ozOox1dLd7e5pnI7comoOsDOTvKDVQyJoscpR84p+mSrs5OBmygumCMnIoo5BnIiophjICciijkGciKimGMgJyKKuYoDuYjcICIvicifReT/BXFRRETkXUXDD0XkbwFcCmCuqr4rIqcGc1lERORVpS3ybgDfUdV3AUBV36r8koiIyI9KA/mZAD4hIk+LyGMi8lGnA0VktYj0iUjf4OBghU9b37iwAxH5UTK1IiIPAnifzV3rRh9/MoDzAXwUwH0i8n61mfevqpsBbAbMFP1KLrqeWasCWetwZrPjK/pwpiUR2amo1oqI/BuA21X1kdHb/wXgfFV1bXKz1oqzIGqQE1F9qlY98l8C+NvRJzgTQAuAvRWeMxaqlf5wqvnNWuBE5KTSQL4FwPtF5D8A/BTASru0Sr0pXhTZSn8EEcwrXbCCiBpPRYFcVY+p6gpVna2qH1HVh4O6sChbt248h205csTsr1QQqwIRUWPhzM4yVDP9EcSqQETUWFiPvAzt7fYdkkGlP1gLnIj8YIu8DEx/EFGUMJCXgekPIooSplbKxPQHEUUFW+RERDHHQE5EFHMM5EREMcdATkQUcwzkREQxV1H1w7KfVGQQgM2UmpqajgYp8FUhvk/e8H3yhu+TN07vU1pVTyneGUogjwIR6bMrB0mF+D55w/fJG75P3vh9n5haISKKOQZyIqKYa+RAvjnsC4gJvk/e8H3yhu+TN77ep4bNkRMR1YtGbpETEdUFBnIiophr6EAuIneIyEsi8icRuV9E3hP2NUWJiFwsIv8pIq+IyE1hX08UicgZIvKIiLwgIn8WkbVhX1OUiUhCRJ4VkX8J+1qiSkTeIyI/H41NL4rIolKPaehADuC3AGar6hwAuwDcHPL1RIaIJADcBeAzAM4GsFxEzg73qiLpOIC/V9WzAZwP4Dq+T67WAngx7IuIuA0A/k1VPwxgLjy8Xw0dyFV1h6oeH735FIAZYV5PxJwL4BVV/YuqHgPwUwCXhnxNkaOqb6jqH0d/Pgjzn+70cK8qmkRkBoBlAO4J+1qiSkROBHABgB8BYwvc7y/1uIYO5EVWAfjXsC8iQk4H8Gre7T1ggHIlIh0A5gN4OtwriazvA/gmgFzYFxJhMwEMAtg6moK6R0TaSj2o7gO5iDwoIv9hs12ad8w6mK/ImfCulOJMRKYA+CcAX1PVt8O+nqgRkUsAvKWqz4R9LRE3CcBHAGxS1fkADgMo2T9V90u9qeqn3O4XkS8DuATAEuWg+nyvATgj7/aM0X1URESaYYJ4RlV/Efb1RNRiAJ8TkaUAWgFME5FeVV0R8nVFzR4Ae1TV+lb3c3gI5HXfIncjIhfDfNX7nKoeCft6IuYPAD4oIjNFpAXAFQB+FfI1RY6ICEw+80VV/V7Y1xNVqnqzqs5Q1Q6Yv6WHGcQnUtW/AnhVRD40umsJgBdKPa7uW+Ql/AOAEwD81vx/xFOqem24lxQNqnpcRK4H8BsACQBbVPXPIV9WFC0G0AXgeRHZObrvFlV9IMRroni7AUBmtAH1FwBXl3oAp+gTEcVcQ6dWiIjqAQM5EVHMMZATEcUcAzkRUcwxkBMRxRwDORFRzDGQExHF3P8HhI5qZsMGLXQAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KVNer3vnpDxm"
      },
      "source": [
        "At the initialization phase, we just randomly initialized the weight of $G$, as a result, it certainly does not match the training data. Our goal is to setup a generative adveserial training to get it to close to the training data."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MsI-zBsgp9tC"
      },
      "source": [
        "### Discriminator $D$\n",
        "\n",
        "Now let us build a discriminator network $D$ that classifies the real data from the fake one. Here we use a three layer neural network. Additionally, we make use of the Softmax loss to measure the classification likelihood. Because we are only classifying two classes. Softmax function becomes the sigmoid function for prediction.\n",
        "\n",
        "\\begin{equation}\n",
        "    \\frac{\\exp(x)} {\\exp(x) +\\exp(y)} =\\frac{1}{1 + exp(y-x)} \n",
        "\\end{equation}\n",
        "\n",
        "We simply reuse SoftmaxLoss here since this is readily available in our current set of homework iterations. Most implementation will use a binary classification closs instead (BCELoss)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "oettjmbLp9a8"
      },
      "source": [
        "model_D = nn.Sequential(\n",
        "    nn.Linear(2, 20),\n",
        "    nn.ReLU(),\n",
        "    nn.Linear(20, 10),\n",
        "    nn.ReLU(),\n",
        "    nn.Linear(10, 2)\n",
        ")\n",
        "loss_D = nn.SoftmaxLoss()"
      ],
      "execution_count": 13,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "AtcTUZertq3D"
      },
      "source": [
        "## Generative advesarial training\n",
        "\n",
        "A Generative adversarial training process iteratively update the generator $G$ and discriminator $D$ to play a \"minimax\" game.\n",
        "\n",
        "\\begin{equation}\n",
        "    \\min_D\\max_G\\{-E_{x\\sim Data} \\log D(x) - E_{z\\sim Noise} \\log(1- D(G(z))\\}\n",
        "\\end{equation}\n",
        "\n",
        "Note that however, in practice, the $G$ update step usually use an alternative objective function.\n",
        "\n",
        "\\begin{equation}\n",
        "    \\min_G \\{-E_{z\\sim{Noise}} \\log(D(G(z))\\}\n",
        "\\end{equation}\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mwzkIy2VsxZe"
      },
      "source": [
        "### Generator update\n",
        "\n",
        "Now we are ready to setup the generator update. In the generator update step, we need to optimize the following goal:\n",
        "\n",
        "\\begin{equation}\n",
        "    \\min_G \\{-E_{z\\sim{Noise}} \\log(D(G(z))\\}\n",
        "\\end{equation}\n",
        "\n",
        "Let us first setup an optimizer for G's parameters.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DFEqvNyJpEcP"
      },
      "source": [
        "opt_G = ndl.optim.Adam(model_G.parameters(), lr=0.01)"
      ],
      "execution_count": 14,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fCJtUQFUu-mo"
      },
      "source": [
        "\n",
        "![image.png]()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fexOFSSZvjRu"
      },
      "source": [
        "To optimize the above loss function, we just need to generate a fake data $G(z)$, send it through the discriminator $D$ and compute the negative log-likelihood that the fake dataset is categorized as real. In another word, we will feed in $y= 1$ as label here.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "NhPHESKKpCoA"
      },
      "source": [
        "def update_G(Z, model_G, model_D, loss_D, opt_G):\n",
        "    fake_X = model_G(Z)\n",
        "    fake_Y = model_D(fake_X)\n",
        "    batch_size = Z.shape[0]\n",
        "    ones = ndl.ones(batch_size, dtype=\"int32\")\n",
        "    loss = loss_D(fake_Y, ones)\n",
        "    loss.backward()\n",
        "    opt_G.step()"
      ],
      "execution_count": 16,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9Qsn3cFwv-f5"
      },
      "source": [
        "### Discriminator update\n",
        "\n",
        "Now, let us also setup the discriminator update step. The discriminator step optimizes the following objective:\n",
        "\n",
        "\\begin{equation}\n",
        "    \\min_D\\{-E_{x\\sim Data} \\log D(x) - E_{z\\sim Noise} \\log(1- D(G(z))\\}\n",
        "\\end{equation}\n",
        "\n",
        "Let us first setup an optimizer to learn $D$'s parameters."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WOf4qlOvv-JV"
      },
      "source": [
        "opt_D = ndl.optim.Adam(model_D.parameters(), lr=0.01)"
      ],
      "execution_count": 17,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WfBh-6Adwnw9"
      },
      "source": [
        "![image.png]()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "c3gqNBcnxCF0"
      },
      "source": [
        "The discriminator loss is also a normal classification loss, by labeling the generated data as $y=0$(fake) and real data as $y=1$(real). Importantly, we also do not need to propagate gradient back to the generator in discriminator update, so we will use the detach function to stop the gradient propagation.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "H4RwEfQ4nluL"
      },
      "source": [
        "def update_D(X, Z, model_G, model_D, loss_D, opt_D):\n",
        "    fake_X = model_G(Z).detach()\n",
        "    fake_Y = model_D(fake_X)\n",
        "    real_Y = model_D(X)\n",
        "    assert X.shape[0] == Z.shape[0]\n",
        "    batch_size = X.shape[0]\n",
        "    ones = ndl.ones(batch_size, dtype=\"int32\")\n",
        "    zeros = ndl.zeros(batch_size, dtype=\"int32\")\n",
        "    loss = loss_D(real_Y, ones) + loss_D(fake_Y, zeros)\n",
        "    loss.backward()\n",
        "    opt_D.step()"
      ],
      "execution_count": 18,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "S-0El6XyyPNe"
      },
      "source": [
        "## Putting it together\n",
        "Now we can put it together, to summarize, the generative adverserial training cycles through the following steps:\n",
        "- The discriminator update step\n",
        "- Generator update step"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "EeF1zzALyO83"
      },
      "source": [
        "def train_gan(data, batch_size, num_epochs):\n",
        "    assert data.shape[0] % batch_size == 0\n",
        "    for epoch in range(num_epochs):\n",
        "        begin = (batch_size * epoch) % data.shape[0]\n",
        "        X = data[begin: begin+batch_size, :]\n",
        "        Z = np.random.normal(0, 1, (batch_size, 2))\n",
        "        X = ndl.Tensor(X)\n",
        "        Z = ndl.Tensor(Z)\n",
        "        update_D(X, Z, model_G, model_D, loss_D, opt_D) \n",
        "        update_G(Z, model_G, model_D, loss_D, opt_G)\n",
        "\n",
        "train_gan(data, 32, 2000)"
      ],
      "execution_count": 19,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KCqCAuzazp8B"
      },
      "source": [
        "We can plot the generated data of the trained generator after a number of iterations. As we can see, the generated dataset $G(z)$ after get closer to the real data after training."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 282
        },
        "id": "2gWdtbKEs1vW",
        "outputId": "b781a776-4053-463b-ae04-8b52897c0cd6"
      },
      "source": [
        "fake_data_trained = sample_G(model_G, 3200)\n",
        "\n",
        "plt.scatter(data[:,0], data[:,1], color=\"blue\", label=\"real data\")\n",
        "plt.scatter(fake_data_init[:,0], fake_data_init[:,1], color=\"red\", label=\"G(z) at init\")\n",
        "plt.scatter(fake_data_trained[:,0], fake_data_trained[:,1], color=\"pink\", label=\"G(z) trained\")\n",
        "\n",
        "plt.legend()"
      ],
      "execution_count": 20,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.legend.Legend at 0x7fc6354e5b90>"
            ]
          },
          "metadata": {},
          "execution_count": 20
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD4CAYAAADvsV2wAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO2de3xU5bX3f2uGDCHhoky4WJCEvrXeAZGqFFtro+KhVj199QiN1ILCARRtT48eK+dYP74nPVpbfdF6KVgsr0TQ2mq1pfUC3ltsgyIqIlBJEIRAhnJLCCGZ9f7xzE72zOxnz56ZPTN7z6zv55NPZvbs2fuZ29rPsy6/RcwMQRAEofgJFHoAgiAIQn4Qgy8IglAiiMEXBEEoEcTgC4IglAhi8AVBEEqEPoUegB1VVVVcU1NT6GEIgiD4hrVr17Yy8xCrxzxt8GtqatDY2FjoYQiCIPgGImrWPSYuHUEQhBJBDL4gCEKJIAZfEAShRPC0D9+Ko0ePYvv27ejo6Cj0UEqK8vJyjBw5EmVlZYUeiiAIGeI7g799+3YMGDAANTU1IKJCD6ckYGZEIhFs374do0ePLvRwBEHIEN8Z/I6ODjH2eYaIEA6HsWfPnkIPRRD8R0sE2LoDONIJ9A0Bo0cAw8IFGYrvDD4AMfYFQN5zQciAlgiwqRmIRtX9I53qPlAQo+9Lgy8IguAJUs3et+7oNfYG0ajaXgCDL1k6BaCmpgatra22+/zqV7/CDTfcYLvPq6++ij//+c9uDk0QhJYIsGY98Fqj+t8S0e+3qVkZe6B39m7e33gsEd32HJOWwSeiJUS0m4g+MG0bTEQvEdHm2P9jNc+9JrbPZiK6JtuBewFmRjTx6p1HxOALgss4MeIGdrN3g74h6/PotueYdGf4vwJwccK2WwGsYuYTAKyK3Y+DiAYD+BGAswGcBeBHuguD2zQ0ADU1QCCg/jc0ZHe8pqYmnHjiifjOd76D0047DZ9++inuuecefOlLX8KYMWPwox/9qGffyy+/HGeeeSZOPfVULFq0KOWxH3vsMXzxi1/EWWedhbfeeqtn+/PPP4+zzz4bZ5xxBi644AK0tLSgqakJjzzyCO677z6MGzcOb7zxhuV+giCkgRMjbuBk9j56hDI+ZgIBtb0ApGXwmfl1AHsTNl8GYGns9lIAl1s8dTKAl5h5LzP/A8BLSL5wuE5DAzB7NtDcDDCr/7NnZ2/0N2/ejHnz5uHDDz/Exx9/jM2bN+Ovf/0r1q1bh7Vr1+L1118HACxZsgRr165FY2Mj7r//fkQimqUhgJ07d+JHP/oR3nrrLbz55pvYsGFDz2Pnnnsu1qxZg3fffRdTp07FT37yE9TU1GDOnDn4/ve/j3Xr1uErX/mK5X6CIKRBOi4YJ7P3YWHgi9W92/qG1H0fZ+kMY+adsdu7AAyz2GcEgE9N97fHtiVBRLMBzAaAUaNGZTWwBQuA9vb4be3tantdXebHra6uxjnnnAMAePHFF/Hiiy/ijDPOAAAcOnQImzdvxle/+lXcf//9eOaZZwAAn376KTZv3oxw2PqDfvvtt/G1r30NQ4YokburrroKmzZtAqBqD6666irs3LkTnZ2d2lx4p/sJgqChb8i5cR89Ij4DB7CevQ8LF8zAJ+Jq0JZVR/SsuqIz8yJmnsDMEwzjlynbtqW33SmVlZU9t5kZP/zhD7Fu3TqsW7cOW7ZswbXXXotXX30VL7/8Mv7yl7/gvffewxlnnJFxdfD8+fNxww034P3338cvfvEL7XGc7icIgoZ0XDAem707wQ2D30JExwFA7P9ui312ADjedH9kbFtO0S0Qslw4xDF58mQsWbIEhw4dAgDs2LEDu3fvxv79+3HssceioqICGzduxJo1a2yPc/bZZ+O1115DJBLB0aNH8etf/7rnsf3792PECPWFW7p0ac/2AQMG4ODBgyn3EwTBIeka8WFh4JwxwHkT1H8PG3vAHYP/HAAj6+YaAL+z2OcFABcR0bGxYO1FsW05pb4eqKiI31ZRoba7xUUXXYRvf/vbmDhxIk4//XRcccUVOHjwIC6++GJ0dXXh5JNPxq233trjAtJx3HHH4Y477sDEiRMxadIknHzyyT2P3XHHHbjyyitx5plnoqqqqmf7N7/5TTzzzDM9QVvdfoIgpIHPjHhaMLPjPwDLAewEcBTKD38tgDBUds5mAC8DGBzbdwKAR03PnQlgS+xvhpPznXnmmZzIhg0bkrbZsWwZc3U1M5H6v2xZWk8XTKT73guCkH8ANLLGpqYVtGXmaZqHai32bQRwnen+EgBL0jmfG9TVZRegFQRBKBak0lYQBKFEEIMvCIJQIojBFwRBKBHE4AuCIJQIIo8sCIKQLh5qapIOMsMXBEHQYSWVrFHUfPO3EVeFGnOBGPwMaGlpwbe//W18/vOfx5lnnomJEyf2aOYAwLvvvotrr71W+/z3338f3/3ud9M6Z1NTE5544gnLxz777DNcccUVKY8xZcoU7Nu3D/v27cNDDz2U1vkFoeTQSSVv3mapqDmqa4frQo1uU/wG32V9ZGbG5Zdfjq9+9av45JNPsHbtWqxYsQLbt2/v2efHP/4xbrzxRu0xTj/9dGzfvh3b0hD1sTP4n/vc5/D000+nPMbKlStxzDHHiMEXBCfopJK7uy13HzkkXnTNEGr0EsVt8HOgj7x69WqEQiHMmTOnZ1t1dTXmz58PADh48CDWr1+PsWPHAlCz6nHjxmHcuHEYNGhQj8bNN7/5TaxYsSLp+E1NTfjKV76C8ePHY/z48T0NTm699Va88cYbGDduHO67776k55x22mkAVKesb33rW7j44otxwgkn4JZbbunZz+i0deutt+Lvf/87xo0bh5tvvjnj90IQipo0u1Jt252sqJmtUKPbFHfQNgf6yB9++CHGjx+vfbyxsbHH+AJqVg0Aa9euxYwZM3D55apdwIQJE3DXXXfFGWQAGDp0KF566SWUl5dj8+bNmDZtGhobG3HXXXfhpz/9KX7/+9+nHOO6devw7rvvom/fvjjxxBMxf/58HH98r3bdXXfdhQ8++ADr1q1L67ULQkmhk0oOBJJm/szA7/88MGlXN4Ua3aC4Z/i50kc2cf3112Ps2LH40pe+BEA1MkmUdW5tbcX06dPxxBNPYNCgQQCUYf/ss8+Sjnf06FHMmjULp59+Oq688sq4RihOqa2txaBBg1BeXo5TTjkFzc3NGbwyQShxdFLJREm7EgGXTjoQt81toUY3KG6DnwN95FNPPRXvvPNOz/0HH3wQq1atwp49ewAA/fr1i9Oh7+7uxtSpU3H77bfHzfw7OjrQr1+/pOPfd999GDZsGN577z00NjaiszP9Zsd9+/btuR0MBtHV1ZX2MQSh5NFJJWt8+McP7UR1tTL+1dXAokXe0/EqboOfA33kr3/96+jo6MDDDz/cs63d5DY6+eSTsWXLlp77t956K8aMGYOpU6fGHWfTpk1xFwCD/fv347jjjkMgEMDjjz+O7tiXK1H7PhvcPJYgFDVWUsma1oZUHkJTk/L2NDV5z9gDxW7w6+rUZdbFyy4R4dlnn8Vrr72G0aNH46yzzsI111yDu+++GwBw0kknYf/+/T0G9ac//SlefPHFnsDtc889BwB45ZVX8I1vfCPp+PPmzcPSpUsxduxYbNy4sae71pgxYxAMBjF27NikoG26hMNhTJo0CaeddpoEbQUhXTzWmDwdSMkne5MJEyZwY2Nj3LaPPvoorjmIF7nvvvswYMAAXHfddZaPHzlyBOeddx7efPNN9Onjn7i5H957QcgFDQ0q12PbNuURXnZvBOd+zpuVtkS0lpknWD1W3DP8AjF37tw4P3oi27Ztw1133eUrYy8IxYiTMh2r7O7J08No+Lv/umLJDF9wjLz3QjFhGHJz5nZFRbLXt6ZGGflEqquVr95ryAxfEAT/YqVn4wJ2ZToGDQ3Wxh7wXlGVE8SnIAiCdzH0bIxCJ0PPBsjajZKqTMdYAejwWlGVE7Ke4RPRiUS0zvR3gIi+l7DP14hov2mf27M9ryAIJYBOz2brjqwPnapMx2oFYODFoionZD3DZ+aPAYwDACIKAtgB4BmLXd9g5kuyPZ8gCCWETs8mTZ0bK5bdG8Gorh0YOaQT23aHcNviEfjdX8I9htzOZePFoionuO3DrwXwd2Yu6lr+XMojr1u3rkd/Jx2cSiQ74dVXX8Ull8i1WfAAmiIn7XantERw7pBmjBrWqTJ0hnfi0Vua8cLjkR5DrlsBVFf709gD7hv8qQCWax6bSETvEdEfiehUl8+rx+WAT67lke0Mvp1EglOJZEHwFbkqcrJwFVX0jarc+hg5KNQvOK4ZfCIKAbgUwK8tHn4HQDUzjwXwAIBnbY4zm4gaiajR0KfJGF0DgyyMfi7lkTs7O3H77bfjySefxLhx4/Dkk0/ijjvuwPTp0zFp0iRMnz5dK5/sVCL5xRdfxMSJEzF+/HhceeWVOHToEADgT3/6E0466SSMHz8ev/3tbzN+fwTBVXR6Ng4Ctg0NQFWVKrInUrd78uztXEWxiWEOCvULjptZOv8E4B1mbkl8gJkPmG6vJKKHiKiKmVst9l0EYBGg8vCzGpFdwCfDCH8u5ZFDoRDuvPNONDY24uc//zkA4I477sCGDRvw5ptvol+/fmhvb7eUT07ESiK5X79++O///m+8/PLLqKysxN133417770Xt9xyC2bNmoXVq1fjC1/4Aq666qqM3htByAnDwql/rwk9Zt/8bARmzgzDrD0YiQAzZqjbdf9LI30MAEc60f5eM2bdA7y1Wfn0HRt5j/e6ddOlMw0adw4RDSdSmqJEdFbsvO4k09qRw4CPgdvyyFZceumlPcqaTuWTrSSS16xZgw0bNmDSpEkYN24cli5diubmZmzcuBGjR4/GCSecACLC1VdfnenbIQj5Z91GYOPWuJX8lwdvxc/mNAEAptVGsHXFenSvbsSmx9fj7ecj1q4iExV9o6i/bkd8z6RU7mEf9Lp1ZYZPRJUALgTwr6ZtcwCAmR8BcAWAuUTUBeAwgKmcjxJfXQODLAI+p556Kn7zm9/03H/wwQfR2tqKCRNUYVu28shWGAJqQLx8cjQaRXl5ueVzrCSSmRkXXnghli+Pvy5LIxTBl7RElKG3IEDA9f/ciu9MjqBvGdA3pMxNzfBO/M+MZgDVyjVkzMYtGDVUbW9vB95+PoK6USnqATQeBdXrVu1jXECAwriGXJnhM3MbM4eZeb9p2yMxYw9m/jkzn8rMY5n5HGb+sxvnTUkOAj65lkdOJV2sk092wjnnnIO33nqrZ3xtbW3YtGkTTjrpJDQ1NeHvf/87ACRdEATBc7REgI+bbHchAgZWco+xN6gsN7l1zxkDBIOWzze3LPy3f3ZQD6C5cHip121xSytkEfDRkWt55PPPPx8bNmzoCdomopNPdsKQIUPwq1/9CtOmTcOYMWMwceJEbNy4EeXl5Vi0aBG+8Y1vYPz48Rg6dGgmb40guIvZhWL8Ga6UrTuUklmmGMa5JZJsyAF0dAK3Le6dGBqzfe1xAK3nwEu9bkU8LQeIPLIgZEmipILLdHcDO/8Rwohwp1XHQuzZF8TQy8/oub91xXrUDNe4h88Zox1z+5EArvtJNZavip9k5lJ4TcTT8ozIIwtCllj5w10kEABGVlkbewAID4x3ld62eATaOuLNZVtHAG9+ZnIPxzwKh7pCiDLQtCuE7z1Ujadfjzf2hczl96XFYWaQ7pPyAOXl5Zg+fbr28RNOOAEnnHBCHkeUPV5eCQpFQGI6o4uZdFakMh+Jbhhjhv7jWTswamivFMOft4RRf7i3OcrgwWEcPBifDhoKAeEwsHevqt5NK83TZXxn8MvLyxGJRBAOhz1t9IsJZkYkEtFmBAlCVlgpYqaBMRdxyxwwA5Xl3ZhWG4lzxSxfFU5yzQDxmvoRi2Tzzk6gf3+gNanqKP/4zuCPHDkS27dvR9ZVuEJalJeXY+TIkYUehlAsmGf0WdLdDbjhHWXurcodckw3Ft+s0i6tjLxBMKhX1DTjFe183xn8srIyjB49utDDEAQhU9IJyKZw77R1BNAvlL2v3zD2ZirLo/jxrB1YviqMcBg4cAA4erT38YoKZ8Ye8I52vgRtBUHIL04DskYGzHkTgOOqejYzoycoOuueasu0R7cYNbQTFRXAwoXAY4/16uqEw4DDuklPCa6JwRcEIb84ceOYCyRbIsCuXuc4EUBQxnjh/G0YPOBoVin5bR0BtO7XF1+1twPXXAMYeRhz5gCHD1v76w369/em4JrvXDqCIPgcJ1k4AZN/Zcu2pCIrs689U5iB5pZQT4HV4pubVRVujLaOQM9jRkF7czNgKrLXEg4DNgXzBUMMviAIeaOhAXj7+RH4nxnxxhWAsuCGYe/qRvSjrdj0t4M4aUDmRj0Vo6eOibufmHZpF7C1wytB2kTE4AuCkBeMpuDt7WG0tvYa1/ZoCP3Lu4GueMMeIODE/q2WAVWn2D030Y2jS7vMBK8EaRMRgy8IQl4wNwU3G9fqaqBpaXJPByC73PojnYSyPmx5DGbgpgdyY5W9FKRNRIK2giDkBSs3x7TaCP628F3Xz8UMLP59WJvB07o/aKlvU1vbK55pI5cfR//+ymfvxSBtImLwBUHIC4abw9yQZNmCrVkFXnUQAfMub8XH20KWGjiJs/vKSuDQIWD1amDkSGDZMuD4452dKxpVaZvRqBJE86qxB3yolikIgj9paABefiKCRf+2FWV5ciZHo8BDz1bhki8fSCsYm05RFZBb9ct0sVPLFB++IAh5oa4OuPJzTSjLo18hEAAu+fKBpGycVLS3K9eO0/5CXs3KSURcOoIg5I1QIP8eBW3zkhR0d6uZvhMCgcL2qnWKzPAFQXCfRLnj0SOy6jSXDZlKL4RjwzVcO4GAXhGiu7uwvWqdIjN8QRDcxRBHM6ppj3SqZuOv5T8eZ5Y6TpdIJF4+IRhU2vY6Ctmr1imuGXwiaiKi94loHRElfbKkuJ+IthDReiIa79a5BUHwEJu35bRbVTqYpY4zMfpmjh4FBgxQAVodXvfluz3DP5+Zx2kixP8E4ITY32wADhQpBEHwFS0R55HOPGJIHWfL3r0qG0dn9L1aYWuQT5fOZQD+HyvWADiGiI7L4/kFQcg1m5oLPQIt5uCtuRZg64r1jmf/gwer//X1yQFdL1fYGrhp8BnAi0S0lohmWzw+AsCnpvvbY9viIKLZRNRIRI3S1UoQfMSmJs+4cqwwgrfTaiNYfHMzaoZ3IhAAaoZ3Onb5HDigsnHq6lRFraGP7/UKWwPXCq+IaAQz7yCioQBeAjCfmV83Pf57AHcx85ux+6sA/AczayM5UnglCB7HxVaFuaStI4BZ91Rj+aowtq5Yj5rhyeNt2hVylK/vpSIrK/JSeMXMO2L/dxPRMwDOAvC6aZcdAMzFyiNj2wRB8BM+MfIGXV3oMfaAPi/fab6+1wOzdrji0iGiSiIaYNwGcBGADxJ2ew7Ad2LZOucA2M/MO904vyAIeSIx5dIHBALxjch1efnG9lBIaen4NTBrh1s+/GEA3iSi9wD8FcAfmPlPRDSHiObE9lkJ4BMAWwAsBjDPpXMLgpAvnPajLQA673Sigb9t8QhLQTWju1Vnp8qn92tg1g5XXDrM/AmAsRbbHzHdZgDXu3E+QRDyTMyNwx2dWWnU55KubqCzK6BtU2hgzPbtultt29YbgF2wQN0fNUoZe68HZu0QtUxBEOxpiaBrQzP6BLw5szdgBh58Jn1lTCu8Hpi1Q9QyBUHImEMf7ED/Pt429oBKj8xEGTMRv7tt7BAtHSEzGhqAmhoVEaupSU8qMJvnCvmjJQKsWY/KoHWA1ovOAbtMm9pavfql4abySz59psgMX0if3m7U6n5zs3OpwGyeK6RHNoqVRjZONKr12XvRl2+njLllizLmCxaor52hd19d7X/fvFPEhy+kT02N+sUk4sTxmc1zBeeYDHYPgQDwxWpnRn/Nel+lXgLxxVU6PGzuXMPOhy8uHSF9dJUnTipSsnmu4Byr9MloVG13go+MPbOqkk1l7I3m5KVMaRn8Uvcdz5sH9Omj1uJ9+qj7Tkh83wwFqUScVKTo9vFzNYsX0Rls03bjY627MILtv1kPfrVRzexbIsoF5COcZON4UMQz75SOwTd8x83Nakpg+I5LxejPmwc8/HDvt767W91PZfSt3reDB4Gysvj9nKY2FGM1ixfRGezYduNj/fIXIlj0780YWRXLrz/SqVxBgweiK+oP80AER9LHMsMvJR9+qfuO+/SxnuIEg0psRIfufQuHgf79M6tIaWgormoWL5LCh298rDohMbNZ8GJwNpFoFAh+3dJtHYeHzZ1riA8fEN+xbj2bap2re3+MThDRqPqfjsGuq8v8uXaUusvOzLCwMu7GTL9vKC5ga3ysujRGot4/L+FUPsEKu05VpULpGPxC+Y69YoR061mr7eYxBzRfEa/53EvdZWfFsDBwzhjgvAnqvyk7x/j4Igf84+eIRlUlrZ0Ojg7xGipKx+AXwndcaCNkNtz9+lnvMzuhV03imK1WAMb75pWLGaBcREZuv4Efukq7jJOPpKEBOHRI3e5b5v0K2h4ImH9/DWbdU42mXSG1QEyRneOn5iT5oHR8+ED+fceFjBskFjgBvX58ZjWznz0beOih+OfpxhwMqimW8b4BycevqCjcLysQ0K/3S6SyxuojT/xIEveJvtLoObeNjq4u9TGno5GzbFnRf+xJiA/fIFe+Yx2FjBtYzXi7upTBZla3E4293dii0fj3LRcz6lTTU7vH7VxMbq+svLSyMaH7SF5+QqVdRl9pxKQ+63HZRGf9W70Es5qvpNuScObMPAzOR5SWwc83hcw5T3Wx0RktuzGbn2O1CrA7bypSub+sHp8+Xa3Za2qAKVP0QimAe+6dQrvpbLB666fVRvDzG1XapWEsG/5zK/b/YS0euLEp72PMBObk4HFledRRKmZnpyc+Gs9QWi6dfONkjZ0r7NxJ9fX6cQHWj11zDbB0afIU0ur4ZneVUzdaKveX7vHEMa5cqd+PKPvmHR5O77Uami7tErA2pF6DGWAAAYtxOk3F9MBHk1fEpVMoCtna3i5IbeeOSRxzOKwCvg8/nNrYm4PgDQ1AVRVw9dXOZsOpViSpVg7t7crYNzXltjedh9N7rfIPRg3TSyR43dgbsOYa7SQVE/DER+MZxODnkoYG4Kabeg2ekRqRD+wuNqmMlhHrePxx4PBhIJLCV5p4fGNlY/U8nWtFZ4wDAXU8J8baGH8uM7K8JA0Rky/Ga0oSobpfBIGAcuNsXbEe3asb4RObroXIOnPYSSqmgdcyiAuJGPxMcJr7NnNmvNGLRIAZM+L3z2UAUBektjOu5nFYrQQSqa5OPn6q51ldcKyMNKCyimbPVj76VBivK5crq3QuJrn8bBObiR/pxBmVzVh4QxMW39yMmuHKZ++XWbyBEw9zVxdSCqUZSP59PGLw08Vp0G7BAhUxSuTo0d4ZbjoBQDeNh51xNY/Dzmdu0NysrEpVVe+YUq2hrS44hpG2ms4Z7pqwzQ88FIr/ZVtd7Nx4D51eTHId3LVQw6wsj2LuZa1xPV2LkUAAjox9MCj594lkbfCJ6HgieoWINhDRh0R0k8U+XyOi/US0LvZ3e7bnLRhO0xHtjJ7xmN2xzMapqkqtDNwyHobRsjOg7e3pqU2ZVy92a+jEKZfxOolU0FUn9WBcfBJF2wxSTQ3dNMBO0nuzTFtNdW3iDmvfvK4w2g8wAwfbUy9JnPjuidRXyfgpCQo3vh5dAH7AzKcAOAfA9UR0isV+bzDzuNjfnS6cN/dY/eqcBu3sjJ7xmO5Y5hk2szKmR4/G7+NGmuHhw/aPd3fbpzomYqxedCuIcNi6Csgw5ql0fSIRvcDL0aMqTVOXu3/11akNsJurqCyCu06uTTsi1kbPby4cMwfbCXPurUmSTjCj892HQr3zF6Le67+HsmY9QdYGn5l3MvM7sdsHAXwEwFk0JRdk+qNNfN68eda/Oqda8PX16luYSFmZ8kfX1OhnpcFgat85oDceuvfAvP2aa5z5583uCyfTx23brN0ey5YBra3xs2EnMYJEOjv17xtzfIwk8YKiG2/ivm6417II7jpZHPzHIyOSDKOHM6xTcqRTGfvlq8Jx0gl79gWxZ1/QVkahuhpYskR9vaqrk9+HElTY0OJqHj4R1QB4HcBpzHzAtP1rAH4DYDuAzwD8OzN/qDnGbACzAWDUqFFnNjvxIxtkmvdu9TzzNMFMZaXanrjvnDnJlatGlo4RuA2HgX/5F/t89ooK50bQKsFY9x44zaM3M3du/GtyMn1MJ+nZTg4hG4wxVFWlzjBKledvVVfg5DtmtR+gvgMLF9p+H3Vvi7mM4MaqBvzXuD+iatZVwNBhaN3djb4DQhhY6U+r39EJzLx7tCPfvJnEj8fJe1fs5CUPn4j6Qxn175mNfYx3AFQz81gADwB4VnccZl7EzBOYecKQIUPSG0SmflOr5+kMUVsbMHFivPFjVsY0caZXV6d+3MZMt39/4Kmn9EbXmFHb+dYNdOkHuvdg0aL0Z9NPPaX+GzPaVJSV9YqqVVX1ul/MAV0zutVStmzbps6Xytib30PdxKK5OX7FYLUyam9X280zfl2cJBJJ6WNIuThoaMC9B2djyKoG0OIHQbtbUDU0iP4+DtaWh+KbmJhTS7euWG8po2D1E/BS1qwXcWWGT0RlAH4P4AVmvtfB/k0AJjBzq91+aVfaZnp5T3emabS7T8TJbFCHeYy6makxTruKVbdnzXPnAo8+mhxDsKK2VrlUZsxI3j8UUutu85j791cX0HQIhYABA+yNuVF4Zbc6NCqOAXWRdFLF63SFZJ7xZ1CZm3IRYRyzdjJw8wKgXKOE6jOMytlptREsvrk5LtsosUG5Tg+vkMXtXsFuhp+1wSciArAUwF5m/p5mn+EAWpiZiegsAE9DzfhtT562wU9nWW4u9z90KPVs0AmJFxbdeKwwxtjQoAKMTo5vRTrnzDfmz8HudRoEAsmvt6wMuO465xchKwy3CuD8gmw1FjuM15pqEpL4XZwyBVi5Ety8DTuCo04lnDwAACAASURBVPAf3fV4q7qu17i1RIBX/wwMHaae36eP8zF5nKZdIYyeOkYrB2E8nupnUOoN1XJt8M8F8AaA9wEYH8NtAEYBADM/QkQ3AJgLldFzGMC/MfOfUx07bYOfiT4soIwIUXzefEWF2mY1A3U6w3c627bTsTHjpK1gOquKfGP+paa6MIXD9hfhcBjo6Eh/hWBg9/m6heHOsXodOk2jRMzfX6u2hUWCeQbfvbrRMj/AWAGUmjZOuuTUh8/MbzIzMfMYU9rlSmZ+hJkfie3zc2Y+lZnHMvM5Tox9RjgpirHycR89qtwEic/7xS+sqypnz3ZWbalzHIbD1mO0y1oJhYADB/RqkYZP2PweeA1z3zw7Yz93LrBvn/2xIhF1gZ47N7PX2t6eW2MPqDHu369v+O4kS6m9HXj+j8Ca9eCPthadsWdOzr7R5dlv2x2SytksKT21zHT9/Lr1oZN1o13GzMqVyc+1WxGkmvFaZQql495Jdfx8osuQ8iu6lZmTFWCR+ekT6eoCyi6In4zqfPjff7ga510ZLin3TCbk1KWTS3Ji8DMIoqWN+WJgZKLs3dvro00M/hnLdl3wMB0DWFkJlJer81VWOhdsq662HpuQPbrJhJML8orngOHH5WRYXiDKQPD8ZNs0rTaCH8/agVFDO/HZ3hCaaATO/VZ6KZulisgjm8l1b9vEAp5IRFW0Pv64uqCsXKlPHa2vt851T+ei3NamzpmuOmdzs5JANiSR/VyymW/CYfuiNJ1rT1eRbGbosMzH5QOimuLq5avC+Py0MQicPwEj//cYMfYuUXoG300lRauKy1S1AHYl93V1hXdlmC8YgjMiEb1v3RB1s/qu2AnGGexuycWIPUMm10khc0rPpeMWOv+8zh1iLOtTuZS8nFYppI+uurqsDBg4ULneBg9WAXlzmmntZGDW9b0zfD+rotlgpFomQqQWxeKvTx9x6eQC3UxeN1szpiupXEpOlvmCf4hErLuFHT3au5JKFMe78WZgwZ3Kd2/0KChC7JqYMIuxzwXF+U3KFU6aeFspTBL1NvBI5VJKfLxIf+xCMgyomf3lVxT1587M4F070X7PvcCqP1nu48Ws4mJAXDpOcVrQZGS7PPJIvB880/puJwJgQtHAK54DFXFWDgBg105g6qUAgDZUYBYWYTl6fxdWKhyCc8Sl4wZOimQM18zKle5ptO7d62y/QCC9hiWCNymSrBztPLLjMLD4wZ67lWjHjxH/uxgwQIx9rhCD7xQHjSvQL1YcY6e86BTDfeR0BRaNpm4gInifA/sLPQLXaNoVQpRVcVU0CvCuncA99cCqF+L2G4X435YsaHOHGHyn6HLEzPnqhvStLoc9cQZu16jEaU9ZoXionQwaMLDQo3CN0VPHIHj+BJRdMAHBr09A89T5ScYeALYh/rdFlIcOVblsMO9hxOA7xSp7xqoCtr1dPys3z8DtOixl0glK8DdGZk6RuOWi+w+gGwFsRQ2mQRnT21CPNsT/htpQgdsQX/TInOMOVbluMO9hxOA7xSq7Jt2AdzDYO6O46SZ9gVYqbXahuCiyzBzu7ETwgZ8gAEYNmrEYszENDViOOszCIjShGlEQmlCdFLA1cOJBTYtU7T1LpA9icXzD8kVdnSqOikbVf13uWDhsbZi7u3tnFDpHZXOz3iVkpHAWySxQiDHr+uIx9sygu++Mc91YBWbV9kNYiJuSVgKAy1W2iTN6XazL9auM9yiOb1mh0BVRLVyYeS69TiiNqFdlcelSmekXE0WSmQNm4JlfW/rpR2EbpqEBizEbNWhGAIwhiGAIIkkrAdclkJ26SEtAy0Hy8LPFiUyyUyGysjL7Lk7mz8o4rwR2/Y/fFTGZ1ar3d78B338PrL7texBGOLAPAZ1aWoztwWq8trTJ3bRMJzLURdQHUeSRC40Tg6/romWgk2+Wwiz/UztZBWz96NaJRoFnnwbuv0e7yxGEECBGGTtoSemkjWe66PSpgkF1riLrg2hn8IunIaaXcdJcJFUOvW6NK8bev9ROBub/ABh0TKFHkj7MQMsuVURl4cIx6EIQXDkAZW0Ov6e5cKtYtZIsohl9OvhwSuFDFi7MbvYWDsd/Mc0ZB4K/qJ2sXDir31az+mOO7W376CdadgFTLwWvegGMmA6QBX0oivJ2h9Xiuepf6KYkus9xxWIQ0cVE9DERbSGiWy0e70tET8Yef5uIatw4ry9oaFApmJkuU40gsPl45owDwT8Y7Qr9roJpkkcwLlNRS889lPSzk1l7ro1wYoZdCRp7wAWXDhEFATwI4EIA2wH8jYieY+YNpt2uBfAPZv4CEU0FcDeAq7I9t+dJJbgWCNhfCKqrk32LUpTlX2Zd7+/etBo3DgEI6ub4R46kPq6b7UUFW9yYYpwFYAszf8LMnQBWALgsYZ/LACyN3X4aQC2R39awGZDKOOuMPRGwbJn1TESycvyL39Mv9+9TKpc2PvskDh1KHWdqbi45iYNC4YbBHwHgU9P97bFtlvswcxeA/QCKv0llpoUcVt0fDL+94F/83q5w4KDcHTsdiQMrHZwS1cZJF885EYloNhE1ElHjnj17Cj2c7LDzXVZUqGCsFYkVvCKmVhwsflD5v/3Kgf29AecVz6mYhNukkjiw0sGZMQOYObMktXHSxQ2DvwPA8ab7I2PbLPchoj4ABgGwXOcx8yJmnsDME4YMGeLC8AqIrl1hOKwCVAsX2rc7NBC/fXGw6gUlD7zvH/4LuHd2AhWVvQHn4cepALSd0c/Ua6tbGTc0WOvgHD2qxmemRLRx0sUNg/83ACcQ0WgiCgGYCuC5hH2eA3BN7PYVAFazlyu+3MIqHWzZMqC1VT3mNF2sBDQ+SoZVLwCXX1ToUaRHVxfQ3qZaUZkp76cC0Toy/Ykzq99DMAj0768uMFVVytin0/NBfjdJZG3wYz75GwC8AOAjAE8x84dEdCcRXRrb7ZcAwkS0BcC/AUhK3SxajHSwxx9X96dPj/cxOkkXKwGNj6LHnH//h1cKPZr0CAT0/vtUgehshP6iUaCtrbfRe7oNftL53ZRIDMCVSltmXglgZcK22023OwBc6ca5fElieqbhYwSc5QNbVQqGQqoXnFUGhF26Z0WF+kJv2GD9eCLhMPCPf7hf7l5KGPn3RkpmZf/CjiddjGCzld5PqkB0vrqwhULxbp10irisfp/TpwNvvQU89JD7Yy0gngvaFiVWPvh0fIxWrp8lS5RriFm5icyPHXus9XGCQXWcDz9Uz5s71/68oRBw8KAY+2zxc/59Z6cKNlsFnBP60xaUJUsyr6S1+n0yA488UnQZQCKelg90an25EIpyej6d2mYopIJggweLTk82JOrk+KnsxPju7N8HPPCz3rz72snq4jV0mJrZp9DRyRtG4da8ecrQd3eryc3s2c5m6HZqmuEwcPiwr3R4RC2z0OjU+nJVYZjqfHYVwIYefyr1TkFP7WTgP25PDnL6hbZDwDfOL/QonNGnDzBokH5yMneuMvpWMuZA5hLjHq4OFrXMQqNT68uFUJTufGVlquoxELBfWRgTADH2mTP/B/419gDQzyfNdfr3V9INdivRRYuASZOSffQzZ6rvul3/CTt8mgEkPvx8kG+1vsTzhcPKwEcivc0qBPepnQw8+6I/5Y7NeLkiOBhUs3Zm9b1OZbC7u6199J2dqZ9rVxzp08w5cemUAtIkJfckZuL4lY7DqjjMC755K8z2ykknK6PJSTp2jkjFsAD1u0lsO+pjH77M8EsBMfa5pXYy8MM7/G3smYFdO71t7BNxMsuePTu92Xh1taqZOXy493djFIIZj3vY2KdCDL7Qi58ySbyCMbPv4/NwWKyhiaeNfaJ7pb5eHysJBHoDtlYSJ6GQimuZMeJqujRNI1DrU2MPiMEvDXR+yMR9Hn88u8rIUsTPOfYG0ah38untMDcCApThHTDAet/jj1fB2poaVURF1NtwJhgErr0WeOwx67iaLiDr00CtGTH4pUDiDyURY3Y6fbpk5zjFkEoYNrzQI3EHL8/szSQWQenclUY1u6Gg2dbWm6zQ3Q0sjbXnsJI10bmAfBqoNSMGvxSoq9PP8o00TSODR0jNjTerfrTDjysON5iXs3LM3HRTsjSyHXYKs+ZK98SLyJQpzlRsfYgY/FJBJ8V87LHOcpErK1NLMZQCtZOBy6/wbz/aRLwkj5CKSMRdmfBt26z19ZcuVcqcRdj0XNIySwmrasPp053P7InURaKtLbfj9DIrnrMWEfMbzMnSCaWGXTW5hytpUyHSCoIenQyDHWVlmVco+p3Vb/t7du9nQ2+nAus2udK5ygOShy/oqa9P3w89cGByG8ZSwS/+7s5O1VkrGlUNTIw8+//+L9WAxW/GHsjeABvxKieZaIkB2iJRzBSDX+rU1aUfrN27Vy13ly2zbuFYzPihL21XF3D3ncqwf/1s4IKJwPlneT/PPhXV1dmlDR97rEo9dnLhMHSnamqUCmein9+nPXPF4JcCqWYn6c7WjdmPWbMH8Lerww5zt6pZ1wMrn/duRlPHYeB/7vC3YddRWZld2nAkogy1IZuQal/DuD/yiPN+Fh5fCYgPv9ixkkJO1ALR7TNxIrB6dXo6Ik70TYBkfRKvcuPNyVk5HYeBjg7gGE2jmXzT3a3eTy9p1HuZcFitUrP9/iX6+Z381vKA+PBLGSfdtnRqni+/rJbA6aSn6YpTwuH448yZk707KNc58DfeDPzzlckrl/J+qserVy5Y3d1A/e3+d9lki5OKckAZ+zlzsj9f4nc92852eUBm+MVOvrttpTPLMaeJeu17WDtZFVfZuanMolqFZtdOZfBLGadNe4yUy3RUZJ0oZub7t6YhZzN8IrqHiDYS0XoieoaILIXAiaiJiN4nonVEJBY8n+S7TDwd7f+6OvXDe/zx3IwlG2Zdnzom4RVjD6i2g6WOE2Nvrpjdu9fZcSsq1Iog1XfaB5IM2bp0XgJwGjOPAbAJwA9t9j2fmcfprjxCjrBSCsx1mbhhyBM1ShJpaFCzrKuvzt1YMsWrBlS3EvJLumghqayMN9RO3I/hMNCvnwrcAmpyovtOp/qteSCgm5XBZ+YXmbkrdncNgJHZD0lwlWy6beXyC2q4fvKg1Z+Rs8jLBjQxLdRP8giFJPFiaWWgzY+ZdfGdpGPa/dasJBwKkNrpmg+fiJ4H8CQzL7N4bCuAf0D99n7BzItsjjMbwGwAGDVq1JnNmTQYFrIn1xkHmVT4GhjBOQcXiyMowwEMxBCkeWHxagerXTuVcZ91vVqFlHJmTiaZXkYHLENa5K23gF/8ItnHXlGhZvZW37FMZBd03/ccSDhkJa1ARC8DsNKAXcDMv4vtswDABADfYosDEtEIZt5BREOh3EDzmfn1VAOXoG0ByfUX1Gn6phUVFUrc6uGHtbswgLZwNeYfrMeRTmAxZqMSaQhv1U5WzciN/rRGgLaQfvtoVGXjlKJxTySfMguJZBKEzWNAN6ugLTNfwMynWfwZxv67AC4BUGdl7GPH2BH7vxvAMwDOyvC1CPki100g7AJZds2jAbXqWLlSvw8R3pq7DMMON+FXnXVYjjrMwiI0odqZe8eY3R9zbK+RN8ryC0U0Cjz7tBh7g0Lq3GQShHUa0M2xnz/bLJ2LAdwC4FJmtpw+EVElEQ0wbgO4CMAH2ZxXyAO5zjjQ+U/DYeU2spJzNrNtG7BwIbpC8fswCJgzB1evrIvzRi1HHUajCXVYhjakyP/3UhcrZiWV8OzTwP33FHo0/kLX/jBbDh1K3xA7SZ7Ig58/2yydnwMYAOClWMrlIwBARJ8jopWxfYYBeJOI3gPwVwB/YOY/ZXleIdfkOrvHKsC1bBnQ2qoeMx7XaaeMGoUG1GEWq5l7FIQmVGNm2eNomPSQdiFinu1HQehCwvFrJ3urixWR6kg25ZtqbIJzBgzIjcifIdGQjiF2kjyRh8ItKbwS9Fjp5+e7CYRN8LhmQZ02zAA4iwk/gHm4Hg+DAO8Gag2kuCp9qqvVjDydbDCnSQFuB1xd8vOLtIKQGU7z6XM9Bs3MyC7MMGWK/WGnoQFbUdNr7AFvuXKs8GptQC7JRh0TUFf9gwdVDwcnlJUpd2IqlyLgflPzPBRu9XHtSIKQKwwXTwKjRlnP4gcPBn75S/3hpqGhN2undrIy9F5y4+jIVW1AOJyXeoiMyEYd06CzUxVdOWnac/SoWtUaM/cFC/RLRbcraOvrrVezLhZJygxf8C26MAOgfuM6Ftb+EZUrVgCv/DW+GbkXpBJ0LtZoNDfFVaXSz6Ctzbm4WnOzypAB1JfManUQCrlfrZ5NkaRDxOALvqafyQNjJPjYSaQ8cGMTqhbc2Gvkvabhb3XRyWVKZnt7YWb32bhqgkEV4M8kIOv0AmdkyNx0k/XKYMAAXzY199i3XRCcYaXMcDimOGCstA0/fTcC2IoaPFD7HOZd3grympG3o6tLFVsVU0pmRYX68Jz61RPp7lbGtr4+vdTLvXvtM78SsbsYOhVeS4c8pGVKlo7gS+wKgevrgReuacDD3fHVtbziOdDw4/I3SDeIRlWbwmyorQX+8pfklD8z+WpIY3xAhr7MTTdlvsKorlbR+aeecnYMI6smmyrvxGO5iUvV7ZKlIxQddhk6dXXAQ8csSJJSIK9muTDrg5PmQK3h7knHHdK/PzBjhprZ2vmwzQYw28wYHYbhMlwhdXWq7iJT42u0H4xEeus47F6j4XNPJ9gaDudPbTbX1e0Qgy/4lFQZbP33WvxIvKqASQQcPGCvglldrdQbmdOTFTh0CJg5U91ubXXm+84kM4YIOOUU/cUilZF0GlBNxLhYGO4Pu5n+ggVqVZGOK2jhQqXbZLyuYFDdz4X/Pg9pmWLwBV+SshDY6key+EH79J00iEYZ3B0FGwY4WxfBwEHAPfWquCoaVf/vqQf+8oYy0uaZcboGoLNTuU6A3toKtzOSmFUmTFeXum1cWJxmmyxcmOzTTzfW0t5uvzoxLgoAsGRJ6ouM8fjSpb0Xwe5udT8XssZ56F0hBl/wJSkz2Orrk3R2sOoFoL0t63N3dzGo6ygoGAC5Jay2txWYMR147Y/AhV9WFbWvvmw9m6yvT/98kYhqNmMYqlx0YTL7n9Mt2qurAx57LN4IH3usij+k81q7u+0zcQypArM7adkya0O7cGF++9TmIS1TgrZC0dHQALz26wjqp21C1dAgsLsFZGjGr34761RMZlaG3sF+3M0I9Elxvo4OoG0vcPig8x4E8+Yp/3W6v1/jeID7ncaCQTXDzxSdjMY11yh11Obm1MFlIyhsVzBlJVWgkxHxSJ/adMhKD7+QiMEXnGD+rQ4eDEyZEMHiHzShb6j3u82dR8F334norPnoM3xo3sYW3bcPgY7DShbhwH5wWQhknk3u3wc88DPglZdiT7AwIrosDeOFOzGEVsdzOnNO59jZ2BNdlko4rGbjgP1rTrw4upH1ksfGJW4hWTpC0ZKYuhyJAD+buy3O2AMAhcoQmf9jfGfx2WjryN/XPjBwoHLPfP1s4IGfgYKB3qpeIqC8XO0YjepnjOYsDbNe+oIFaibKrAK6TguRjOM53Z9ZBTnDYTVmnZ88W2VKXTZKJNLrijJcRebXbO49O316r468Gz5x3TGmTCl4f9pMEIMv+BorF2vVIOssk6pB3Vi+KoxZ91Rn5XkAlLvGEebMICtxtvJ+arsdhr+9oUGlWJoLc2bMiDc2hvGzK2oyjjdlivNZfmenSvGMRlXQMhfBRbu4gpXP3DD+ut6zQPY+cSu/+sSJyp1W4P60mSAuHcHXWLlYo680WtoxZiBwvlrpdq9uzNiVf7QLOHqkGxWV9vnq3NkJuvvOXkkEXfzArrjK7KaoqrJOO6ysVC8u8coXCiVnJZl9+In+8lSY/da5kM6eN0/fttLOZ54Lt4vu9TU06GMfHnHz2Ll0RC1T8DVWipmt+4MYckzyLL91f6+B3rY7hJrhmaVo9gkCqz8chHPHtKOyXB+4o2BApVUa7G5RGj6J6OoDgsH4Gakux7xNk3l09Cgwd64KeCYarpqa9Iw9ED8D1yiYZsXKlfrH7Gb/bhcsJQaPzSsGI73VCicNGAqMuHQEX5PoYp1WG7GMMXZ0Ajc90Gs0bls8Akc6M0ulJAIunHAIj60cjKZdIX2cMtgn3h3wzJPJs9SOw8D/e9S6EOiYYzIaXw/MyohapUemawxzVV1qxm5Miec2xzJ0S7VMU0/tUjHtCrtyVaHsImLwhbyQq97MZhfrty+I4NFbmlE1qLvHpcMM7NkXxMy7R2P5qt4c7+WrwuiOZu7ODASAS758AKOnjkFzi6Zqs28oPh99xTLglP+lthuPjzsF+MPvrAuBElvp6QqF7HxTOiPqxBgGgznLB0+ioUH/OsLh+HMnRuqtKoOzuUBlumJwQ7s/x4jBF3JOrkUA6+qAprcjaPjPrajoGz+DJgKOdAXjjD2gZJL79c3uvKOGKpfQbYtHJGf+BALA6BHJTxoWBs4ZA5w3Qf0fFu59Ef37J+9vLvJZuDB5JRAKAf/6r/rgq86w65rIG1RUqOBsPrqdGV8QneFeuDB+m9UMHHDvAmUncWBXnZuL/rkuIwZfyDm5KFY0rxhunBpB1wa9/3REuBPTaiPYumI9ulc3YuuK9Uom2aFHR+eyaY+GUFGBnsyfpl0hRKPAoa4Q8MXqXmPulFQzy7o6tRIwZ4wsWQI89BAwZ06y0beb5SZmn4TDvWmX+ZjRm7Ez4Fbj0L1PRmprthcou3TOhQutXTdlZbl3eblAVlk6RHQHgFkA9sQ23cbMSZEXIroYwEIAQQCPMvNdTo4vWTrFgdvFiokxta0r1tsHYPsE0XaI4wKszC7IyZw0Gg0vh91LVsk228QLTeczId0vSD6Koezey0RZ53BYXQg88l7nrNI2ZvAPMfNPbfYJAtgE4EIA2wH8DcA0Zt6Q6vhi8IsDt3+ficdLmWIZDGbmX+0TBLpsnnee5W8qc3TSAvmcbReCdL8g+XyffHgRLXSl7VkAtjDzJ8zcCWAFgMvycF7BI7gtAphoG7bttpG6JUrL2DNDBVNPGg1MOiOzAWZKHsSzPEm6X5B8vU956ECVb9ww+DcQ0XoiWkJEx1o8PgLAp6b722PbLCGi2UTUSESNe/bs0e0m+Ai3f5+JLlTLoKlBmivYtu5QfDC1jybVTrc9W9JVmSwGMvmC5ON9yqdSZp5I6dIhopcBDLd4aAGANQBaATCA/wPgOGaemfD8KwBczMzXxe5PB3A2M9+QanDi0hGssPK9T6tVWTrZ+OW7o0BwRBWw9wBwpFPN9AcPBHZF4i8cRMCJNekHZQV/4UOlTCBLlw4zX8DMp1n8/Y6ZW5i5m5mjABZDuW8S2QHgeNP9kbFtgpASq/x9q+y35avC2BFJo6F1IkTK2LfsVcYeUP9b9gLDw/G582LsS4M8dKDKN1m5dIjIXCf+zwA+sNjtbwBOIKLRRBQCMBXAc9mcVygNdC7UKVOsXb5NNCK5eMepYA4RsPsfyTO3aFTN+K1y54XiJg8dqPJNtj78nxDR+0S0HsD5AL4PAET0OSJaCQDM3AXgBgAvAPgIwFPM/GGW5xVKAJ0LdeVKa5fvud8Kq/x382zcfN+OaFQf3D2SmeaO4HOKMIguapmCZ3HNhdoSATY1Z+537RsL5AqCDyh0WqYgZIRrLtRhYWczfZ0Q1+CBaZ5QELyJGHzBs7jqQjVr2OhUDYmAYYOTt7fsVasEQfA5YvAFz5IzF6rOV9/drQK0iUSjwFZJLBP8jzRAETxNLvpsoG/IOhCr2w5I4FYoCmSGL/iajHT2R2vSN0eP0Pv5nWT6CILHEYMv5JxcNT/JWOokMYjb1yRnbHcxEASfI2mZQk7JpbBhzlRyWyLKZ2/IK4weIcVWgm/ImTxyrhGD739yKV3uU6kTQcgpkocvFIxM24M6oQilTgQhp4jBF3JKLo2yOU/f3MLwg1+ul7x5QbBA0jKFnFJfb+3Dd0N/yogBvP18BP8zo7mnhWH/QKeSUgDE9y4IJmSGL+SUXOtP1dUB939vR1y/WgBSLCUIFsgMX8g5OSmeMiPFUoLgCDH4gj8xp07qkGIpQYhDDL7gP5zIHUuxlCAkIQZf8B9bd9gbeymWEgRLxOAL/sPOjXOeZb2JIAiQLB3Bj4jAmSBkhBh8wX+IwJkgZIS4dAT/YfjmReBMENIiK4NPRE8CODF29xgA+5h5nMV+TQAOAugG0KUT9hEExwwLi4EXhDTJyuAz81XGbSL6GYD9Nrufz8yt2ZxPEARByBxXXDpERAD+BcDX3TieIAiC4D5uBW2/AqCFmTdrHmcALxLRWiKabXcgIppNRI1E1Lhnzx6XhicIgiCknOET0csAhls8tICZfxe7PQ3AcpvDnMvMO4hoKICXiGgjM79utSMzLwKwCFANUFKNTxAEQXBGSoPPzBfYPU5EfQB8C8CZNsfYEfu/m4ieAXAWAEuDLwiCIOQGN3z4FwDYyMzbrR4kokoAAWY+GLt9EYA7nRx47dq1h4joYxfG6DWqABRjAFtel7+Q1+UvnL6uat0Dbhj8qUhw5xDR5wA8ysxTAAwD8IyK66IPgCeY+U8Oj/1xMaZwElGjvC7/IK/LX8jr0pO1wWfm71ps+wzAlNjtTwCMzfY8giAIQnaItIIgCEKJ4HWDv6jQA8gR8rr8hbwufyGvSwMxS+ajIAhCKeD1Gb4gCILgEmLwBUEQSgRfGHwimk9EG4noQyL6SaHH4yZE9AMiYiKqKvRY3ICI7ol9VuuJ6BkiOqbQY8oUIrqYiD4moi1EdGuhx+MGRHQ8Eb1CRBtiv6ebCj0mNyGiIBG9S0S/L/RY3IKIjiGip2O/q4+IaGKmx/K8wSei8wFcBmAsM58K4KcFHpJrENHxUIVo2wo9Fhd5CcBpzDwGwCYAPyzweDKCiIIAHgTwTwBOATCNiE4p7KhcA7Ve3gAAArVJREFUoQvAD5j5FADnALi+SF6XwU0APir0IFxmIYA/MfNJUCnuGb8+zxt8AHMB3MXMRwAlz1Dg8bjJfQBugRKXKwqY+UVm7ordXQNgZCHHkwVnAdjCzJ8wcyeAFVATD1/DzDuZ+Z3Y7YNQxqMoWoUR0UgA3wDwaKHH4hZENAjAVwH8EgCYuZOZ92V6PD8Y/C8C+AoRvU1ErxHRlwo9IDcgossA7GDm9wo9lhwyE8AfCz2IDBkB4FPT/e0oEsNoQEQ1AM4A8HZhR+Ia/xdqAhUt9EBcZDSAPQAei7mqHo1J1GSEJ1oc2ilyQo1xMNTy80sAniKiz7MP8klTvK7boNw5vsOJgioRLYByHzTkc2yCM4ioP4DfAPgeMx8o9HiyhYguAbCbmdcS0dcKPR4X6QNgPID5zPw2ES0EcCuA/8r0YAXHTpGTiOYC+G3MwP+ViKJQIkKeF8vXvS4iOh3qyv1eTGNoJIB3iOgsZt6VxyFmhAMF1e8CuARArR8uzBp2ADjedH9kbJvvIaIyKGPfwMy/LfR4XGISgEuJaAqAcgADiWgZM19d4HFly3YA25nZWIU9DWXwM8IPLp1nAZwPAET0RQAh+FwJj5nfZ+ahzFzDzDVQH+p4Pxj7VBDRxVDL6kuZub3Q48mCvwE4gYhGE1EISiTwuQKPKWti3el+CeAjZr630ONxC2b+ITOPjP2epgJYXQTGHjGb8CkRGb3DawFsyPR4npjhp2AJgCVE9AGATgDX+HjWWAr8HEBfqEY3ALCGmecUdkjpw8xdRHQDgBcABAEsYeYPCzwsN5gEYDqA94loXWzbbcy8soBjEuyZD6AhNvH4BMCMTA8k0gqCIAglgh9cOoIgCIILiMEXBEEoEcTgC4IglAhi8AVBEEoEMfiCIAglghh8QRCEEkEMviAIQonw/wE4YRpwJ2fM2gAAAABJRU5ErkJggg==\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "prU6W4Hd19pj"
      },
      "source": [
        "### Inspect the trained generator\n",
        "\n",
        "We can compare the weight/bias of trained generator $G$ to the parameters we use to genrate the dataset. Importantly, we need to compare the covariance $\\Sigma= A^T A$ here instead of the transformation matrix."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Jcnu6eCgtR0Y"
      },
      "source": [
        "gA, gmu = model_G.parameters()"
      ],
      "execution_count": 21,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ZwnY5BWTtdP1",
        "outputId": "05f66e53-4261-4626-c540-b71b5a53f511"
      },
      "source": [
        "A.T @A"
      ],
      "execution_count": 22,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[1.04, 1.9 ],\n",
              "       [1.9 , 4.25]])"
            ]
          },
          "metadata": {},
          "execution_count": 22
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "EV8e1ov5tlkM",
        "outputId": "077e6d67-14b7-45c9-a6e7-4602196e34ba"
      },
      "source": [
        "gA = gA.numpy()\n",
        "gA.T @ gA"
      ],
      "execution_count": 23,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[0.80765074, 1.8812442 ],\n",
              "       [1.8812442 , 5.0232944 ]], dtype=float32)"
            ]
          },
          "metadata": {},
          "execution_count": 23
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "QwYq79Pkt1FV",
        "outputId": "f79d1c73-5be3-44c7-8221-6729621dc503"
      },
      "source": [
        "A"
      ],
      "execution_count": 24,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[ 1. ,  2. ],\n",
              "       [-0.2,  0.5]])"
            ]
          },
          "metadata": {},
          "execution_count": 24
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Htjcsj4vt2L-",
        "outputId": "7933208f-1f2a-4114-8ccd-ec2051f304d6"
      },
      "source": [
        "gA"
      ],
      "execution_count": 25,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[ 0.23406495, -0.22800203],\n",
              "       [-0.86767757, -2.2296433 ]], dtype=float32)"
            ]
          },
          "metadata": {},
          "execution_count": 25
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7K_sWQEI2Wlk"
      },
      "source": [
        "We can also compare the mean"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "00oNrAke2bc4",
        "outputId": "f8cb0329-dd2c-4742-e1f1-31b4f9f5f6ef"
      },
      "source": [
        "gmu, mu"
      ],
      "execution_count": 26,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(needle.Tensor([[2.00028   1.2296444]]), array([2, 1]))"
            ]
          },
          "metadata": {},
          "execution_count": 26
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0HKfa2d_0EtO"
      },
      "source": [
        "## Modularizing GAN \"Loss\"\n",
        "\n",
        "We can modularize GAN step as in a similar way as loss function. The following codeblock shows one way to do so."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "L0EolNkR_XeQ"
      },
      "source": [
        "class GANLoss:\n",
        "    def __init__(self, model_D, opt_D):\n",
        "        self.model_D = model_D\n",
        "        self.opt_D = opt_D\n",
        "        self.loss_D = nn.SoftmaxLoss()\n",
        "\n",
        "    def _update_D(self, real_X, fake_X):\n",
        "        real_Y = self.model_D(real_X)\n",
        "        fake_Y = self.model_D(fake_X.detach())\n",
        "        batch_size = real_X.shape[0]\n",
        "        ones = ndl.ones(batch_size, dtype=\"int32\")\n",
        "        zeros = ndl.zeros(batch_size, dtype=\"int32\")\n",
        "        loss = self.loss_D(real_Y, ones) + self.loss_D(fake_Y, zeros)\n",
        "        loss.backward()\n",
        "        self.opt_D.step()\n",
        "\n",
        "    def forward(self, fake_X, real_X):\n",
        "        self._update_D(real_X, fake_X)\n",
        "        fake_Y = self.model_D(fake_X)\n",
        "        batch_size = real_X.shape[0]\n",
        "        ones = ndl.ones(batch_size, dtype=\"int32\")\n",
        "        loss = self.loss_D(fake_Y, ones)\n",
        "        return loss\n"
      ],
      "execution_count": 31,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "yXQZw5NwAosQ"
      },
      "source": [
        "model_G = nn.Sequential(nn.Linear(2, 2))\n",
        "opt_G = ndl.optim.Adam(model_G.parameters(), lr = 0.01)\n",
        "\n",
        "model_D = nn.Sequential(\n",
        "    nn.Linear(2, 20),\n",
        "    nn.ReLU(),\n",
        "    nn.Linear(20, 10),\n",
        "    nn.ReLU(),\n",
        "    nn.Linear(10, 2)\n",
        ")\n",
        "opt_D = ndl.optim.Adam(model_D.parameters(), lr=0.01)\n",
        "gan_loss = GANLoss(model_D, opt_D)\n",
        "\n",
        "\n",
        "def train_gan(data, batch_size, num_epochs):\n",
        "    assert data.shape[0] % batch_size == 0\n",
        "    for epoch in range(num_epochs):\n",
        "        begin = (batch_size * epoch) % data.shape[0]\n",
        "        X = data[begin: begin+batch_size, :]\n",
        "        Z = np.random.normal(0, 1, (batch_size, 2))\n",
        "        X = ndl.Tensor(X)\n",
        "        Z = ndl.Tensor(Z)\n",
        "        fake_X = model_G(Z)\n",
        "        loss = gan_loss.forward(fake_X, X)\n",
        "        loss.backward()\n",
        "        opt_G.step()\n",
        "\n",
        "train_gan(data, 32, 2000)"
      ],
      "execution_count": 32,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 282
        },
        "id": "eYNLN08VxzzL",
        "outputId": "27a8ec12-813e-49fd-e3d0-b210afe2188b"
      },
      "source": [
        "fake_data_trained = sample_G(model_G, 3200)\n",
        "\n",
        "plt.scatter(data[:,0], data[:,1], color=\"blue\", label=\"real data\")\n",
        "plt.scatter(fake_data_init[:,0], fake_data_init[:,1], color=\"red\", label=\"G(z) at init\")\n",
        "plt.scatter(fake_data_trained[:,0], fake_data_trained[:,1], color=\"pink\", label=\"G(z) trained\")\n",
        "\n",
        "plt.legend()"
      ],
      "execution_count": 33,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.legend.Legend at 0x7fc6354da210>"
            ]
          },
          "metadata": {},
          "execution_count": 33
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXIAAAD4CAYAAADxeG0DAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO29e5gU5Zn3/727mWaY4STNyYDTM/lF4wEREI9EEx0TXTzE+Iuv4ECIJhAB0ay7uCp5EzebcZOXvDGYqARURBgBY9Qkm9mogBo16joq4kZZNJmDoBymCacZhmG67/ePZ2q6urqequruqq6unudzXVzDVFdXP91T/a277iMxMxQKhUIRXEJ+L0ChUCgU+aGEXKFQKAKOEnKFQqEIOErIFQqFIuAoIVcoFIqAM8CPFx05ciRXV1f78dIKhUIRWN566612Zh5l3O6LkFdXV6OpqcmPl1YoFIrAQkStZtuVa0WhUCgCjhJyhUKhCDhKyBUKhSLg+OIjN+PYsWPYsWMHurq6/F5Kv6O8vBzjx49HWVmZ30tRKBQ5UDRCvmPHDgwZMgTV1dUgIr+X029gZsTjcezYsQM1NTV+L0ehUORA0bhWurq6EI1GlYgXGCJCNBpVd0IKhYGGBqC6GgiFxM+GBr9XJKdoLHIASsR9Qn3uCkU6DQ3AvHlAZ6f4vbVV/A4AdXX+rUtG0VjkCoVCUSwsWZIScY3OTrG9GFFC7iLV1dVob2+33OfRRx/FzTffbLnPiy++iD//+c9uLk2hUGRBW1t22/1GCbkJzIxkMunb6yshVyj8paoqu+1+E1ghdzsQ0dLSgs9//vP4xje+gQkTJuDjjz/G0qVLcdZZZ2HixIn4wQ9+0Lfv1VdfjTPPPBOnnXYaVqxYYXvsVatW4aSTTsLZZ5+NV199tW/773//e5xzzjmYPHkyLrnkEuzevRstLS1Yvnw57r33XkyaNAkvv/yy6X4KhcI76uuBior0bRUVYntRwswF/3fmmWeykffffz9jm4y1a5krKpiB1L+KCrE9V5qbm5mI+LXXXmNm5meffZbnzp3LyWSSE4kEX3755fzSSy8xM3M8Hmdm5s7OTj7ttNO4vb2dmZljsRjv3bs37biffPIJn3DCCbxnzx4+evQon3/++bxw4UJmZt63bx8nk0lmZl65ciXfdtttzMz8gx/8gJcuXdp3DNl+bpLN569Q9AfWrmWOxZiJxM989MUtADSxiaYWVdaKU6wCEflElGOxGM4991wAwHPPPYfnnnsOkydPBgAcPnwYH374IS688ELcd999ePrppwEAH3/8MT788ENEo1HTY77xxhv40pe+hFGjRMOy6667Dtu3bwcgcuevu+46fPrpp+ju7pbmcTvdT6FQuEddXXFmqJgRSNeKV4GIysrKvv8zM+68805s2bIFW7ZswUcffYRvfetbePHFF7Fx40a89tprePfddzF58uScc7AXLVqEm2++Ge+99x5+9atfSY/jdD+FQtE/CaSQFyIQcemll+KRRx7B4cOHAQA7d+7Enj17cODAARx33HGoqKjAtm3b8Prrr1se55xzzsFLL72EeDyOY8eO4de//nXfYwcOHMC4ceMAAKtXr+7bPmTIEBw6dMh2P4VCUWB2x4HXtwIvNYmfu+N+rwiAS0JORP9IRH8hov8monVEVO7GcWUUIhDxla98Bddffz3OO+88nH766fj617+OQ4cO4bLLLkNPTw9OOeUU3HHHHX2uGBnHH3887r77bpx33nmYNm0aTjnllL7H7r77blx77bU488wzMXLkyL7tV155JZ5++um+YKdsP4VCkRs5JUvsjgPbW4Gj3eL3o93i9yIQcxL+8zwOQDQOwCsATmXmI0T0BIBGZn5U9pypU6eycbDEBx98kCZydjQ0CJ94W5uwxOvrg+PPKkay/fwViqBirNoEhCG4YoWNhry+NSXiegZGgHMnur5OM4joLWaeatzulmtlAIBBRDQAQAWAT1w6rpS6OqClBUgmxU8l4gqFwomlnXPVppmIW20vIHkLOTPvBPBTAG0APgVwgJmfM+5HRPOIqImImvbu3ZvvyyoUCkUamqXd2iqSkrX+KEYxzzlZYmAku+0FJG8hJ6LjAHwVQA2AzwCoJKJZxv2YeQUzT2XmqVoqnkKhULiFE0u7oUFY62bYJkvUjMt8cigktvuMG66VSwA0M/NeZj4G4CkA57twXIVCoXCMnaWtWeyJROY+jpIlxkSBk2IpC3xgRPw+xryGpJC4URDUBuBcIqoAcARALYAm66coFAqFu1RVCXeK2XbA3GIHgHDYQaBTY0zUuXDvjgPNO4UPfWBEWO4eiX7eQs7MbxDRkwDeBtAD4B0A9g1IFAqFwkXq682zUTRLW2axJ5MuJEsYRXvEUGD3PnFwIJWqCHgi5q5krTDzD5j5ZGaewMyzmfmoG8dVKBQKp9TVCcs6FgOIxE+9pe1ZIaFZfvmn7SkR10gmhdh7QCArO71i9+7duP766/HZz34WZ555Js4777y+nioA8M477+Bb3/qW9PnvvfcevvnNb2b1mi0tLXj88cdNH/vkk0/w9a9/3fYY06dPx/79+7F//3488MADWb2+QlFKWKUle1ZI2LwzU7RleJSqGFwhd7mPLTPj6quvxoUXXoi//e1veOutt7B+/Xrs2LGjb5977rkHt9xyi/QYp59+Onbs2IG2LJq+WAn5Zz7zGTz55JO2x2hsbMTw4cOVkCtKmny/8nYWe85kI84epSoGU8idJoxmwebNmxGJRHDTTTf1bYvFYli0aBEA4NChQ9i6dSvOOOMMAMIKnjRpEiZNmoRhw4b19UC58sorsX79+ozjt7S04IILLsCUKVMwZcqUvsERd9xxB15++WVMmjQJ9957b8ZzJkyYAEBMFrrmmmtw2WWX4cQTT8Ttt9/et582meiOO+7AX//6V0yaNAmLFy/O+bNQKIoNs6/87NnAggXZHceTQkKn4uxlqqJZb1uv/+Xbj5xjsfRm5Nq/WMz5MQwsW7aMv/vd70of37x5M19zzTUZ25uamvj000/n/fv3MzPzK6+8wldccUXGfh0dHXzkyBFmZt6+fTtrn8ELL7zAl19+uelrNjc382mnncbMzKtWreKamhrev38/HzlyhKuqqritrY2ZU33Q9ftni+pHrihmZF95Ivs+4Z73Fd/Vzvynt5hffDP1709vMf9PM/Nr74rfX3tX7JcnKKV+5IUYqLdw4UK88soriEQiePPNN/Hpp5/CWMjU3t6O2bNn44knnsCwYcMAAKNHj8Ynn2R2KDh27BhuvvlmbNmyBeFwuK8neTbU1tb2vc6pp56K1tZWnHDCCTm8O4UiWMi+2syGOQSG7JFXPhmHefOifZks2s07kJs1bt7jqTcLpUCphmYEU8jtEkZz4LTTTsNvfvObvt/vv/9+tLe3Y+pU0Z9m0KBBaX3AE4kEZsyYge9///t97g8A6OrqwqBBgzKOf++992LMmDF49913kUwmUV6efYPIgQMH9v0/HA6jp6cn62MoFEFE9pUHdCKvZY/oUv6mDG7FV88D1m1KiWquQ2iMzbbSLwpZ5Jd7QDB95B6Eny+++GJ0dXXhwQcf7NvWqUtIPeWUU/DRRx/1/X7HHXdg4sSJmDFjRtpxtm/fnibsGgcOHMDxxx+PUCiENWvWINFbXmbsPZ4Pbh5LoSgm6utFgNKMPvvNJHukYmAS98zNTPnL5eY952ZbGh72Mg+mkHsQfiYiPPPMM3jppZdQU1ODs88+G3PmzMFPfvITAMDJJ5+MAwcO9AnlT3/6Uzz33HN9Ac/f/e53AIAXXngBl19+ecbxFyxYgNWrV+OMM87Atm3b+qYRTZw4EeFwGGeccUZGsDNbotEopk2bhgkTJqhgp6KkqKsDbropU8zT7DdJ9kjV6Mztudy85+zR3R0HXn0H2NbsWS/zvPuR54Ib/cj94N5778WQIUPw7W9/2/Txo0eP4otf/CJeeeUVDBgQLK9VED5/hcJyDoGkX3jb7ghi16X3C49GgWdWxPGFzzj3a1dXm7t3YjGRAWOK0d1jJMte5l73I+8XzJ8/P81PbaStrQ0//vGPAyfiCkVQMKYPAqnc8lt+Pg49yczuhG0DxsE4G/0rk+KYXJlejdnzvrWFnJNH165YyKUCIaU4WVBeXo7Zs2dLHz/xxBNx4oknFnBFCkX/xRh8/MWGKNrbgZ98ZyfGj+oGlQsr+wsXRDH4NiDeq9Eza+N47M5mGO2tAaEkDv/3TgweE03LfjncE8FdK8fhl09EMWIEUPflOO6auRNVY7rRmYhg8IRxACSWvJ1Qu1QgpIRcoVAEErPg47pNUazbFEVFBTBnDtDYKNwwmgd5Zm0cKxe3Zoi4RkWoG7fMiOPfb2hFZbmwpAcP6MbPv9OMn9/UjPiBMIZWJjGwjPses2yGNTAiF3MXC4SUa0WhUPhOLuX3VkHGzk5g+fJUJajGPXN39gm06TH3RHDb1zL3CYWAEAGjhif6RLwPq2ZYZsMoANE718Ve5krIFQqFr2TbcUMTfbs8DbPHzTJYNDq6Qrhr5TjLfaTIrG6zYRQn1wBfmOxq3rkScoVC4SvZ5GfrRT8X2vaY+6R7eoC5S2NYtykq3ccWWaB0TFRkpnxxqvhZrP3ISwUv29hu2bIFjY2NWa/JaStbJ7z44ou44oorXDmWQuEWMhdJa2umm0U25cfIzNo4mtdvRWJzE5rXb8XMWiGy33s4M7OloyuE21bW4LktQmDvWjkOHV05SKOLeeHZElwhd7lKij1uY2sl5Fal9k5b2SoUQcWqOMfoZnFSkakFNKvHdguf+9hurPqXFux55h08dmcz9h8idCXCYueBEVROiuGcK1NW8rpNUcxdGkPLrgiSSSBpcNEkZC52DwdH2BFMITebyJHn1dDLNrbd3d34/ve/jw0bNmDSpEnYsGED7r77bsyePRvTpk3D7NmzpW1unbayfe6553DeeedhypQpuPbaa3H48GEAwB//+EecfPLJmDJlCp566qmcPx+FwivM8rP16N0sTioyzQKaAyOMUcMTCIWAkcMSSBxj1P2oBtUzJmLBv0Yxb14qPREQYl4zYyLCF0/Ft5bW4HBPyscdPrVG/uIeDY6wwxUhJ6LhRPQkEW0jog+I6Dw3jivFLMk+z6vhX/7yF0yZMkX6eFNTU1oPlcbGRmzZsgUPP/wwYrEYrr76agDA1KlT8fLLL6c9NxKJ4Ic//CGuu+46bNmyBddddx0A4P3338fGjRuxbt06jB49Gs8//zzefvttbNiwQWr5b9myBRs2bMB7772HDRs24OOPP0Z7ezt+9KMfYePGjXj77bcxdepU/OxnP0NXVxfmzp2L3//+93jrrbewa9eunD8fhcIr6upEqqCslwogLHMiZ77xqjH2YlpZnsTaJc148SdbsX973NJd82hjFBO+ZfBxy/K/PRocYYdbFvkyAH9k5pMBnAHgA5eOa47squfi1XDhwoU444wzcNZZZwGAZRvbxx9/3LaNrRlXXXVVX6fEY8eOYe7cuTj99NNx7bXX4v333zd9jtbKtry8vK+V7euvv473338f06ZNw6RJk7B69Wq0trZi27ZtqKmpwYknnggiwqxZs3L9OBQKU+zSBqWP61yjhzdtxaG/xm2zUJwwszYOi+tBGkTC7bJycWufD11GhkvHLK3Qy8ERNuRdEEREwwBcCOCbAMDM3QC8vb+QJdnncTX0uo2tGVrjLMB5m1uzVrbMjC9/+ctYt25d2r5btmxxtA6FIhes27rKH48NiuMLo1L9RwYP6Mby21pxzsmHcMX5B1E1uhtte0Q1pb797C9uacFNV7UjHAYSCWD570Zi0X3VaWu6Z+5OS8vejMryJBq+14x75u7MeE2NDJfOGP97kOtxwyKvAbAXwCoieoeIHiKiSuNORDSPiJqIqGnv3r15vqL7V0Ov29jatZiVtbl1wrnnnotXX321b30dHR3Yvn07Tj75ZLS0tOCvf/0rAGQIvUKRD3Zpg7LHY4lM12hleRILrm5PC1BqlvLM2jgO/OFtLPxaOwYMEJb0gAHAwq+148Af3kZicxOObWxC4oUmxBy4Vcywss6l/VQKkFboFDeEfACAKQAeZObJADoA3GHciZlXMPNUZp5qdFFkjVmSfZ5VUl63sb3ooovw/vvv9wU7jcja3Dph1KhRePTRRzFz5kxMnDgR5513HrZt24by8nKsWLECl19+OaZMmYLRo0fn8tEoFKbYtXWVPT5upLnYGm2zyvIkli1qw8rFrRhamcywtImAoZVJhEJC2EMk97MzA+0HwrbuG806b3tiK66/JO7egGaPybuNLRGNBfA6M1f3/n4BgDuYOVPNelFtbIuPIHz+iuLCrq2r7PHm9VtRPdaZ5cxsHQTNhlserEF7O7BycatlmX4foZCrZfRu4FkbW2beBeBjIvp876ZaAOaRuoCj2tgqFCnM0ga1zJLqamD6dPO0QrOCG6tOr26x7CbhB1/VOMJZYNXHvPBscStrZRGABiLaCmASgHtyOYgfQy6ywUkb2y996UuFW5BLFPvnrihO9IO6ACHi2qnU2gqsXi3SCo29wI0FNy27Ini+abArWStWaH7wG6bvc37h8CkvPFuKZkJQc3MzhgwZgmg0CnLrXkphCzMjHo/j0KFDqKmxKHRQKCywcrMA9vnf2bpbmEWFZVmON7+OXTZZTvDxGplrpWh8AOPHj8eOHTuQd0aLImvKy8sxfvx4v5ehCDA5z7OEyP3OJtuEGZhVX4Nli9owcpjI7vLE9vMxLzxbikbIy8rKlEWoUASUqipzq3vECGDwYPPHZtbG+8Q4GyE+fCSEtUuaTdt867Gyuq1ejxmIHwxjW08VvnBB8QQ6rQhmrxWFQlFUfO5z5tvjcbmIr1zcilHDrUXc6PllBgYPStqKeD4QAYePhDHrtmCIOFBEFrlCoQgQupmW8Y4IRiN9buXM2jjumbszo0pTX51pZ4UzA0eOAoMGpvYli1xxI1rwNRe3S9XobkduoWJBCblCocgOrftob+pHtFJURAIiI0WztrVcba2N7EOLm9NE2Q4ioMK8U4VjchXztj0RR50WiwXlWlEoFNlh0n20sjyJe+aKnGtZG9mKco+CkjY4sfz1dHSF8K+PjTMvyy9SlJArFIrskORWx8Z0Y2ZtPLeZlz7BDNDwwTjcE0GSRU77natiuOT6aNGX5etRrhWFQpEdku6jRKL8PX4wjFHDnTd98wOt98rtK6rSRLsawH3X+bmy3FAWuUKhyI4RQ6UPaS6VnGZeFhDhf2ccPWo+5DloFPenrVAofEc/HOLGK+LoaN5nuX90WAJzl8aQRSdmX9D8+kHKTpGhXCsKhUKgSynUBiU0bIzixhuB7l5PyvdnZwYyjSR7BdzLXG+3qBrdnZ6dYvIZFFP3QxkB+KgVCoXnmAw079jSisZH4+juFnnhe555x1EpfSiU26QeP0iybmiEB0PdC4WyyBUKhTSlsP7bO8EMPPIvzSh3OEkxmURgMlfCId3QCKuh7kVulSuLXKHox2j+7+QRc+GtGt2Ne+budCzigJjWExRI/8YKMNTdK5SQKxT9kIYGYORIYNYs0QulbY+5UrftiVha17Iu2KGQ/DE/kK5F391QNrw9j6HuhUIJuULRz9Cm28d1rl+zqT0dXSHctXKcVOQB66rJYvKRZ6wlHAZOrkl3mXgw1L1QKCFXuIc+T626WvyuKDrMptubTe2ZuzSGdZuiuGvlOHT3ZB6nmCzurBgQBr4wOdPv7cFQ90LhmjeLiMIAmgDsZOYr3DquooA0NIhveVubaDBdX+98fLhm5mkK0doqfgeKfwR5P0OfNy3rUqhn3aYolt/WisiA9EBgMVncWdFjkeA+JhoI4TbipkV+K4APXDyeopBoQtzaKkwtTYidWtVmZl5nZ2mUzZUYWt601qWwemy3uIkaK7oYzv1qZrrdkIoCTEdW5IwrQk5E4wFcDuAhN46n8IF8hTifWV+KglJfL6bbm3UprCxPYum8gP/NBkaE/zscNn9ctj3AuGWR/xzA7QCkl20imkdETUTUpOZyuoiWfqB13B85MjffdL5CLGveHKSmzv2EujpgxQp5rvfQigQSm5vQvH4rZtYK67z9QEDEjyhVjXmi5NyTbQ8weQs5EV0BYA8zv2W1HzOvYOapzDx11KhR+b5sOv01yNbQANxwQ3r6QTwO3Hijs89A/7nJ6qmdCrFm5umpqECgmjoHnGy/Bjv2mmejEKHP1dLwvWYc+MNb2LD5uGAEN5lFAQ8gxPzkmvTgpTFTpURwI9g5DcBVRDQdQDmAoUS0lplnuXBse/pzkG3JEuDYsczt3d3iMav3b/zczDocZSPE2mvlGixV5EZvbxDu6sYFZRGc/7lxaG2Nmn4N9LFsIuC6i8ah4XvNtimEQysZ865q9/69uIW+gCegwctsIXbxMktEXwLwz3ZZK1OnTuWmpiZ3XrS62ny6aywGtLS48xrFilXVBVFmubEe2ecWDvfWWCshLnoMI9cAkfutpQ1qxGLA9OnA6tWZYZA9z7xT9L3DcyJADa+ygYjeYuapxu3BzyPvz0E2K7eHnUtE9vkkk+JfS0txiHh/dZs5wWbkmkZrK7B8eaaIA8Ctv6gq+t7hRhzZngFqeOUGrv4FmfnFgueQ+xVkKwaBqa8Hysoyt0ci9i6RIAQn802JLHUkPUDMgpgy8UsrBOLiL/Lp6QEe2zgSh3sclM1rDa/6AcG6FJvhR5CtWASmrg5YtQqI6m4fo1HgkUcyrWnjhWf69OIPTqrcdAAWNoOkB4hVSb0Z6zZFUTNjIsIXTcVzbw4uWjHvSQIDJtRgzo+qMbh2IvDFqal/MgLQ8MoNgi/kWi5VLCb8wrGY+N1Lt4DfAqP/Zi9ZAixbJi4ozEB7u7mIGy88q1cDc+aYf27FcLcB9G+3WS+WNkPNOPQkzfuj2DGzNo7m9Vsz0gw/X9VdtBWbYQK2vXnI/MEAN7xyA1eDnU5xNdjpB7Igo12A0Q2M2SaAsKStLl7ZBIRzOb7denPNZFEBWds/3S0z4rjta9Yl9ka0ik59MVBHVwirGkdg4dfai1bIASCRBMKDIpkTfEwCv32UUOBTFuxUQp4LfmbK5PLa2Vx43HxvTi4KVkJv9nwj+VxkjGstwtRJq8SkUCg3u6F5/VZUjzX3oxeziAMmawyFUo2t9GPajOj3CzClm7XiB34Wv1i5G2QukWwCm266M+xcUGZ+g9mzxTe1ulrso3ebmZVWu+HSKpaYhwkjRsgfMxNxmctEj6yis9hFHDBZoz6gOSYKnDvR3J1S4oFPJeS54IdfXkMmyiNGyMXI7sKTbYWnUx+63UXBTOg181Nf0dLSkkqLzOZ1nOJ3zMMlZE2w9GJuJuyBx2iBB3jST64oIc+VurqUwBQy51omyoBcjKwuPEZr1K7Cc8ECYTU7sV7t7gTsBNgopl6lTBZxUHXfPuf7LlvUZtoES8srn1kbx+o7mwMx3T4rjBZ4Pwx8ltqftHD4ldkhE2XZN14TI+OFBxDrnjXL3AcdDpuL/vLlmU5bmfVqdtEBgMOHxbGcCLBeTL1yaRVpTn1Dg/wGCUh3o+x55h2MHGZeoRkb043m9Vux/LZWlAVonqaRo92ELqNRbTbBJ8CTfnJFCbkep+LsxCr1UujN7gayESO9FS7DrMJzyRJ55M3MetUuOlFDgCkeF69vlstutX6vXFpF2PBL+xOZ3SABmW6UUcMTUh83kXCzBLWnOLOYWHTDT6ox/+c1qWIg2QSfAE/6yRUl5BpOA15OrFI/gmcyMZo+PfOCYuYTNpJNIFS2PyBEdvDgzO2dncATTwghtuoPbRRT2Z1FPhfMbC4QBboT0/+JzAKYZr3E7QhCMNMIM1D3oxrUzJiIP38UxSXXR1PFQOdOlIuzFvi0269EUOmHGk7T7mT7Aal0PqfHcjvlTTtea6sQx0RCrEn/N66osBdxIJXbFoul1iV7X0TAmjXytVvl0EWjwP/6X8DDD4uujcbnPfaY/Lhu57zb4fHr6U8H7eOS5XwPiiRLz9dtwt79YVRfP7lguQTFjsojt8NprrWVKGlCbXWsNWtSYmsmsvmesU5yrzWRd4q2LiDz2ETATTcBDzyQvgbjBcXu+ERAR0fmY1b564XO58/j9eyu2bI/myznu6cHGBBgf7cTOrpC+McHY3jqlSj27Suq9H7fUHnkdjj1Mcv2I0q5AZykCALOg4bZ4MRtkkjY+6bN1mXmglizJlPE9e/RyQWjs9NcxAFxHDP3ltWdkVfZJjlmtzjxtMn+bLKc71C4+Btc5Qoz0JUI452OGBqejyIeL7r0/qIjOEKej2/S+NwFCzKP5TTgZbafZpVqpkI2KYJGrERB9hnot1sFMDU0H3AsZr+vcV12aZdOLiTZcsMN6e/VLlCbS867k/1yzG5xkqYuvUbIGmAFoAozW5iFK4VOqUH5xZMx67ZoKaT3F4RgCHk+wUOz5z74YOaxgMwMi0GDMo9nZZVqYjB7tnhuNOosRVCPTBRkn8GCBenb7dBa3GqC7BSnqXi5WsNWDt9jx1Lf3ltvtS/Z1xc6OQ1gO9kvx+wWJ4b8zSMa0IxqJBBCM6oxE+K171o5LqNfeDJp/XEFldbdEZx16+S+wGQRp/cXHcE4HfKpvHNiIXZ2CoEAgCNHUtu1NDnjF7quTnx5q6rEWbVkSaagxuPiWGvWpCxXq3prwFoUZJ/BihXZWcDZ+MaN61qwQDhmicTPBQsy97V7j7Lj2zUN0VoQxC0qE43ZJrLPbM6cdAt/zhxn55f+Ig4I/7+2n4VRYWvINzTgZ4fmoRqtCIFRjVasxDzMREN6v/CkSMNDiVnigPCH/+tj49JO/yJN7y9KghHszKfboFVw0kg0ai4UZtkmZkE/s9fRP3fkSLkQ6bNDzMjmfdgRjYp2twsWiLsTO+bPFz/N9p0/P91HbvUeNSorga4ucVEJh8Vn2dho7S7RxFO2j1nA0cr3UFEhBNxs/pnxGE6aeVkEqm13l/j7WxBDDVoytssCoEGjrxvD7gh+9vQ4nHNl1DYA7GVSUhDwLGuFiE4A8BiAMQAYwApmXmb1nKyFPJ/sBKugmFOMF9P+1y8AACAASURBVIxsjqk9t6FBVFE6Ob4Z2b4Pu2yR2lpg0yZnxyorE8cyW2M4LFIoNOwuOGVl4v3qUw01UX3oIfNh0uEwMHy49QUiGhV92bUKVC1rxopssnf0CpLD+WiZtWL8zGovBeYuBI8eg9Y95fiPPw/FFecf7GtV+x9/HopvX9GO8hKoOO/pASJfnio9/Yu0KaVveJm10gPgn5j5VADnAlhIRKe6cNwU2fgmnUzCyRbjvVw2TrqqqpRpIWPECPtAm6zc3Qwie4FyKuKAEFfZN834OlaulVgMGDo0M1+8s1NY5MZpR4Dw6ScS9la+5gbTu7jsyMbNpHe12DlvTYLrdUuq0dIWQrKqGi31DelipD+/ai8FFi8Bxh4PCoVQPbYbC7/WntYIa+HX2jGwzPnSi5lw2NpV4ldLo6CRt5Az86fM/Hbv/w8B+ACAu00NnFbeOZ2EU1lp/jrRqLMLhlUKotlzrfz0kQhw8KB9oM3on7WikI2lraoy9USj4psoE+TW1sxpR2vXmlvoMjo75VOG3aC1VbiOZBcr/UXbLriu//vqL9JzFwLl6UF245+SqHQyVhLJ4pouGFRc9ZETUTWAPwGYwMwHDY/NAzAPAKqqqs5szdfdYUY2FZUy5xtgfy8ne/6cOcKyND7XrrIxW7+5G+4it5g/H5g2LbMk0YjmPhowwNoSLisDvv1te5+5n2gBX/1FRjt/nLh0gL5z8pWn4qjmnRg34igSe/YiPGYUqFRU2gZm4H8Oj8TJV1T7vZTA4HllJxENBvASgHpmfspqX88qO7MJiubrfDN7PmB+TKsLjJX4aWs3q5y0q97UsLpQ5Es2PmYiYcl6tZZCE42KHjLZXLT1EOGVJ/dicmV6+T0nk6BSzC2UYTU4WZGBp5WdRFQG4DcAGuxE3FOyyVfKxflmHHpcX5/evEmWizx9uvnxpk+3z6ViFrfml1ySeu1bb00PMFoxeLCwmvONE5iRjY9ZS8ksFfbtMz9/HObG7eUROOFYZuMrynV+WxAp4f7ghSZvISdxH/gwgA+Y+Wf5LykPvGxHalc0YpXr3thofszGRiHmTm6lN21Kz1E3BgxlmMUJFPZEo9ZVNzLBdhiUHopDOGH0UfMHnQSrg06J9wcvNG5Y5NMAzAZwMRFt6f0nMUE9xq1+1Wbl2nZFSVaZDLLHWlvNW+K6jdYyVitiUtgTj8stY6061uw80c5BmyDwQHSD9uwyf5CoZEo3tbh1Mqk7zQeES74/eKEJRkFQIZEFMmX+aCeta4HiDdyVIuXlouDIC7R8dcC6WsXsPOrND8foMcCe3cCfXwamX5mRpVJKJBJA17FQugupRCba+4FqY+sUmSDLAntaRoxdJozTSlBF8aPluttlGwGpIjAtP1wv2pq5Wkr5hAakmbADI2LggyIrZEJe4h2Nc0DmBtFavxqFWvvC6vt7yDJh9I8pCz242AVtW1uFgEej6BocRfnhuGl+eCkLuC0lPNHeD0rDEecmMh+yvvWrzP9ulQljfMxYwagoPeJxhA7/HUcREe6UfkYyCbQfkMQKVMaKqyghB9KDVocPi2CWHs3yNut6qLrcKyyIIImjKAPv2e33UgoKM/DAMyPx3V9W4WhP/5po7wdKyI1phdo4EmMvcX0Ay42hyk56kwNiHbKWAorio/ZSYP3vgM1viJ+1l2IIOnDkz//Vr2IiySSw6L5qPL4xisUrY/1qor0fqGBnNp3s8unC6PR19YRCwEUXZdfgSuEfkoAmH9gPAKDhx/m0sMKTTALhi1MxuX50DfMUNbNThlWOtzFHWCa8xu1WY8P08ybtAl3JpBLxICEJaNLw40DDhvuzJh+ZWSuCwkTKA+k1SsitOhkaXSgy4dUXf1i5X4zzJgvZpVDhPVYBzX72dw6FgHvm7gQgTnNP52zmM8+3RFBCLhumbDbhXnZ/qM8vt6oANXtM88d70QtFUVjsApol5F9gZiQZOLi/BzL3bNXoVIqhZ3M23YxbBRgl5GZl/dl+4cLhlDUgc7+0tckf27fPUVm3ooipvVRUlNp1siwRErv3InzRVAy7+ly07h5ouk/bnlT2l+udITQrfNas3Of5lhBKyIHMHG/Z8AaZ5ZxIpKwBGSNGyL/IVVViDatXK8s8iGhBzuHHpf7GJWR9Z9B1BKGVv+j79a6V49DZlS4lHV0h3LVSpBi61beuD6OL0gzPbgGKEyXkZsi6KC5blm69O21sVFEhen/IeqXrq0P1x1cEA1nVplaCX0qizgwsrUfbpg8AAIuiDVix5UxU/HSJcC0lk+Bdn6Lr//4MtOmPOfets8Rq4pZGP2sOp9IPZTgZPGElttGocJlUVYlWtVbT6mV/g7Iy533HFf6x+Y2S6VZoy65P0TFjBtaG5+A7w5+wblfg1ch7u+EdXr1uEaDSD7Ml36mvR44Aa9aI58r6kQNyN86CBUrEg0I/qdrkriPYu3IDVmMOvhVebd9zxitftZW17cktQPGjhDwfrPqldHaKST52hT8y56HWNVFR/Ky8H+g64vcqvCWRwNGlS/FvW/4B34g2YkC3wwHXXviqZa7PtWtzM7pKACXk+bBsmfUtdTxuLeLRqPykK/UJMaXEpmeBpfWl/Tc7dBDlm36P+wYvweB9WYizF75qtwbIlBBKyPNlQI6dgLXgqYaxqKG/+FyDhEkflT42PVvaf7Ohw8RPLWbkBNfTVXTk6/osMUr4zPOYhgYxB9Pp7Ew9RgvCrKhBZa0UF1qK4djjhWCPPR5Y8kPglsV+r6xw1F4q0midzCVVVnJBcSVrhYguA7AMQBjAQ8z8Y6v9A5G1YoXZNCCnZNOMa+BAEfAs5Vv2oLD+d0K8jehTDEOh0r4A7/87MOsa4RK0chnm0kRO4QjPslaIKAzgfgD/AOBUADOJ6NR8j1vUOMljlWF2qykLCB09CgwfLoI4pSwQQUDWR0WrJwiHS/9vNGw4cM40+66dra39uu+JH7jhWjkbwEfM/Ddm7gawHsBXXThu8WIVia+okGezmAU3GxrsA6azZpW+SBQ7/STF0BIiUfzkhH7c98QP3BDycQA+1v2+o3dbGkQ0j4iaiKhp7969Lrysj8iCPeGw8AsuWyavDNWjuWicuE6SSft9FO5hDGz++eXSqtDMlWxH1jnJJTfrXqg6GmZFwYYvM/MKACsA4SMv1Ot6Qn19po/crJrMrjI0HxeNwn1qLxUWpyZW2p3S2OOBr13r37qKiVzuTKzuYI3xptZW4IYbhPWvJRJolj2ggqcS3LDIdwI4Qff7+N5tpYuTPFYn6VH9rLFPUWPMSjG6u0p94r3xboM58y6w6wjw0APyY8g+H+3zHDlS/NOs7AULROaX0Zg5diwzG6wfdjTMhryzVohoAIDtAGohBPxNANcz819kzwl81opbOBn3pigMsqyUfgT39IBCIWF1r7xfbNTuULRtH7wHfPKJPwsk6vcuRs+yVpi5B8DNAJ4F8AGAJ6xEvOTIx5cnG2oBmAdAy8rkBUhlZdlZjKVcvJIL2fp+SxAKhYCLzwFmXCUKnDY9K/6v3+aXiAPOC5H6oX/dlW8zMzcy80nM/P8xs0elXEVIvtNJzFw0a9aIYyUSIu1Q/9i3v20u1tEosGqVeK5V/xeNigrgscfkDbv6Iyorpbg+g0gk/XenVaJm38nZs4Ubp4RRbWzzQeYa8aogwunrLVgALF+e7vfU+mPHYsDnPge88EK/v01NQ/ORG/uK9xe6joh+MZue9XslgrVr7ZMFzJB9R4iEoQPkdtwiQeZaUUKeD7K+yF758py8ntZHvbVVpEMmEkK8tRN2wQLr3uj9BX2GiplPuNSDm3p6eoB/v7t4RDwfQ8iqV3k0KtpL22WbFTGqH7kXyHx2Xk0nsXs94wgsLT+9vV201A2FlIgD5n1TFvdmRGg+4QP7/V2jVxhFrutIcYl4JJJyoSxYIGJCROKn3j0iyz23K64r0fmeyiLPB7OeK15e4e1eT2XBOEOWoaJZpgBw5925d7YsZpJJcQeivxMplIhrd4gyBg8W/YX27RPndUdH5j7z5wPTpmV+DyIRcZE6diz7dQUoG0a5VrzCyUg4L19v+nQxgaitTVUeOsVqNFvXESCRBCorC7umQrHrU3HXUUi0PjRmE6/CYSHKZuIsO9b48dkbLBUVwKBB5lONAtTkSybkJWhyFJi6usL61/Sv19AgquBysUL6M3t2y3PGyweV7gWx60gqFlBIZFZ4NCrcfoC4m3RS5ZxIZF9Ip8WIAPM7Wq96phcQ5SMPMrfeqkQ8F1beb30rHeQgJ3NmVSSzaEFbTFkpQLp17FScw+HsYlCaiC9ZItIQBw0SF5ASmyykLPIgYzf8VpGJlq0SZLGWwQw8/WvgL1szM3KKScDNqKpy5i6RuWHMfOQVFcL1qN83Hhfb16wpCQHXUELeH4hGRQCpVF0GTrllMXD110u3qvXAfuC+peL/xS7cQHrxmlkjOj2aL/0BXa8XY2zKbJtZYzotU0UJuaIoiEatrXKtCKKurjQtUCdoFviYseL3Uv0cmIHNz/u9iuzQt3XWRPXWWzPPaaeZYGbxqtmzzfctsYZ1JWqa9BOM/c2NXHyxsDxK1QK1Q58vXuoFPkTibiNIM0SNoltXJ1IQjWgWtJY7TiQE2klrjELXevhEP/2Glwh1dfLeKpWVwGuvpU72/sjchf2r5D4UEmJee6nfK7GHKFXEM3Jk6kIr85NrYq09bjyn9YU9+mKhw4dFQzk9JZKpokcJedCRTSMqL7dP5yorEz0t1q71bn1+UoodDbVBzzJCIefj2PyEWbhRbrzRedDe7nxua8tsmhWPiwtECWaq6FFCHnRkQy727bN+XjgsIvyaFVNqnRBrL7Wv1gvinQqR+LtZVUgG5QIWj2emSuZDVZV5cLO7W7hsrIa8BBwl5KWA2TQimQ9w8GCRqqUJQWursIqmT8+07IOK5hs3K7E3doQMIpEIcOig/EJVTO1oC0VZmXCjyFwzJRbcNKKEvFSpr8/s6QyIk91oBXV3A088kbLsg47MN85c/OJt5zrRGDoMeOZJ83FsflRv5oJbQXjNbWLloimx4KYRJeSlSl0dMGSI8/3j8ZRlH3Qxl7kWil3EAeE2efrXoieKlaAfPCByxuu/L/ZNJsXPYqvetCLfRlXRqPiMBg+2dtHoOypqlNgUobzyyIloKYArAXQD+CuAG5i5RPt/BhA7P7kMu+KMYieZDG7KZSQCnH+BaGy1+Q37i482ki2IaENONm3K/rkVFan0Wzu3yZAh6X5xYxdRLSMGCKz/PN+z/XkAE5h5IsQA5jvzX5LCEU4simxuJ/VpjPoAKiACo0DxW7S1l4oWtdp6g8qYseJ9HDwg32fosMKtxyumTwe2bMntufrccrvzPB5P/55YVXsGlLyEnJmf6x2+DACvAxif/5IUtjidFWo23DkSybRWy8oyi4s0NwuzaD8ai9n7bvMV+nwE2Fj8E2SIxPuoqCztgOaDD+bXL0g776dPt99X/z1xGhANkPvFzfvPGwH8p+xBIppHRE1E1LR3714XX7Yf4tSiMEtNfOSR1OBlbduqVfa3lFa3r9pxbropv8wX5vQ7A83C3vyG+GlV6FKKxT+RiBhNFuSAZj44idV0dop+/E6Gjmv7ywwGvWWf72D1AmM7WIKINgIYa/LQEmb+be8+SwBMBXANO5hUUVKDJfyg0LNCAeeDn/WDL6zWI5sWU1kpAlcXXpw5DNlqQLBsWEQQMlWsSCZFQDNo3QzdYO1aYNYs+/20nkI33ug8L72iwnqyV6EHqzvEswlBRPRNAN8BUMvMjqJjSsjzxI+TLJexdlaDcK2orAQeXmc+/EE24eaZ54Dhx2VuD7qQ+zHRp1iwGw2noZ33I0c6c9Xoe5TLJnv5YSw5wJPhy0R0GYDbAVzlVMQVLmDm+/a6f4SsglQm4naDcK3o6JCnEMq2G/tpaARZxPuLC0WGExHXn/dOsrS0/c2K6PQErNlWvj7yXwIYAuB5ItpCRMtdWJPCjmxF1c3XtTr5gVQTpFmznH0RZciCeb3bM2ylihKasckcvJxwP4hG0897mchqM0NjMWDOnFRHUKsApp2xVGSB0LzyyJn5c24tRJElhZ4V6gQz90uurLzf3Ede6haqVRyglCHK3g135Ej677L6h+HDU1lZTvPHtd/N3C9FmIeet488F5SP3Gf0AUkz/2CuyHz3TgiFgOOOAyZNTQX2tDzqocPSgnxHUYaDGIpR0PlDZT7yoMAM7N7VfwKZZtgNSjHDLNguG04xaJD58bONLfkYCJX5yNWEoP6Gl9ZEPo2Jkkngn+8AJp+TssKHHycs1PrvA5ueBQPoiMaw45/vw8jPjgKPCoM0gd/8PPC1a4PpE++vVrieyspMC9sJra3CCNCPe9tvUlze2Sm/U8z2vJXt72NjLmWR9ze8tCasLHItw2XJEvk+T/4BGDk6c7t2jiYT2NY5FlWRfagYmMoc4K4joK6u4FnkygpPEQr5lw3ilUXuwZ2vJ1krigDipTVhFiAC0oNSVpk1I0aab9emx4QH4KTK9jQRBwAqHwQeNjyPhfsEs0gt7O8iDjgTcS/aLJeVZZ/t5SRrrMAFRUrI+xteplWZZdOsXQu0t6csEavxdPvabV8iqL2wTCmFMvtCMmdO7p05ZS43bbhKNgLrJGuswP1cSulroXCC1znoTlIUZePpqAc9yVxPSSrugT/GxfWHDBwz8rkSNzaKcyrbOEg4bJ0Rk4u1bHeeF9iProS8v+FXDrqTNVxzFW77VQwtuyJZi7LmfSlKkkkk3/wvJHbtDmbfcLeYPz+/8XqtrcI/PWKE8+dUVACrV9tb8m5bywUuKFLBTkVRoVVGz6yNY9W/tGBgpIjM7DzK/ZO7PsUDM/4T8/ArRFCAoF4uedmFwGnZvRWRiHhvx45Z7xeNirs/s9xvM9wsv8+lpYUDVLBTEQj0BsvBzlDf5DOnE9A8RWvpa7bdhtDoMbgCjViB72RWpbpNNJqd1er0mE47DFqRr4gDojHW0KH2++nTGY099s1w01ou8J2vssgV/rA7DjTvBI52AwMjQM04YEwUDQ3Axsfj+OUtragsT1lHnV0hlEeS/gc7EwmgqwuoqAADoAP7RQ77+ReIIqZk0nzo865PkZxxFZIIYwBcEDMrjJ39gkAudxCxmH0BWjYFQ4V2MeaAssgVxcPuOLC9VYg4IH5ubwV2x1F3SRwPL25OE3EAqCgvcI6xTFTCYeBYN/Cj/y1ywIcOEyK+8n7g4nOAf79bBDL19AY2CfBexIHgiXg4LNrQZjtYZPp0+5RErWCouhpYsEC4O4wibuzZEkCUkCsKy+44sK050xeZTAIftgHbWxGSuKF9t8Y1hg0HFi8BjT1eLGrs8aIvTO2lIoC5tN50IHKxxmJdIZ/pTsmkEFGtwtgpjY0iJdEubqHlcS9fbn6RGzzYGxEvYGMt5VpRFA7NEvexn7MryIKe+fYO1/y3TvrVzJ8vsjFk1rcbQUWnVFQIQbVajxWVlcDhw+L/CxYI6ziRSH3GMo0iEn7tXPv76I/j9jmpgp2KkqV5p+UXxvdgZr7IeqU7oaJCuArsRKmsTBRZPfCAEAWzAGRFhRARLyohjWhBPG09WnAvGzo6xHOqq4Fp00RAmVmcKzfdJH9eVZU7edlepASqgiBFyaL5xM0IhfD3w+a35z0JAMdLyvcLTXc3cMCkKRNgPfXeDE3wtD7Zq1fbP0dfiVhXJ6pm165NWfNEQjAefFDc0ruRaSJDCyTqq3a1IplcMCvMaWw035dIFLFlK8LGi4xXA1lUQZCiZBkYkT92UgyLllWhoyv9lOzoCmHOv9cAJ1UDJ9cA+/+ek+met7XPLF77Jz8EfvF/zWdDVlRaD4jWowX4mIX4NTY6d0sYBU/rYVNWlv5GDx8GDmR5cdGIRoX7Rn+B0GMngLleQDo7xVASzacsEz7m1PvO5s7j4otT/vxwWFxAvfCPF7ggSAm5onDUjMuMWIZCQqDHRPHqh1HMXSoqO5NJoGVXBHOXxvDqh72iMCYK3DpXZIwYM0NsyKmOR7u93/WpeM2rvyKCmZueBXd2ZO4fiQDzFqZvKysT2/Vo1YZ6AcnWUjPepi9ZYl4gY5b37oRDh4Sbo6VFfA5r1mSXE71sWeb4vWyi1drFyiofvrpa/NTnh1sFXSsrgddeS8UOEgnxd/AiCFngcYyuCDkR/RMRMREVyf2voigZEwVOiqUs84ERYMwI4Tt/qQl/e7QJDd9rBgDMqq9BzYyJ+O1r0fRzv74+MzOk4zCQSICZkUwCSYP1nbM1vnuXSCk0dCjsQAUwVNJtcfTYdMFbtQp45BF7EczFUtNK1q0s11zp7hb51hpOeujoqasT713/vh97TFj5Tq+q2h2KzOLW99LXLjg9PcLVZCai5eWF81sXuCAobyEnohMAfAWAf13VFcFhTBQ4dyLwxanCQt+9r893Hurtl1I9thsrF7di0XVxce5fEgde3wq81ITkuM+DN78hpghpuduXXwTUnov2Hy3Dke5QWvpiMgkc6szBHDc0tWIASRBaEMNcrEDrnoHmzysfmJ3gadTX53bb4MRyzZVsp/UY0VwfWlByyRJh5WvWPWD/nvfts67INBNimYjKhjN7NRAi24tfHrhhkd8L4HaYzMNVKCyxyGKpLE/ivu/uFCKuKx4KEUDG3G30WslzF2YUEoVCwNFjoQzfuxTNF25oavUxxRBGEjVowTrU4a6V4zKP2XUE+N7torJzwQKxzWlf6ro6kaGRi5hrVqbRlSGjsrIwHcZk7x3IdNnIqKpKCaJszWZCbCaiBfZbF5K8hJyIvgpgJzO/62DfeUTURERNe/fuzedlFaWCVRYLgGRXN1petEhZLB8E/t6/oWf9H7Cqdh2io82FLDo0gVWNI6QuFmaAzXzhvXSgAndwfZ/7dSYacM+mMzFo6RL07NoDThqm3icSImvkkktEMM3p7fwDD6QLm/aCTkR33z7hyrCjogL41a/Sfd6y4+eb8SJLwTNz2cjcIXq/Wr5CXGC/dSGxLQgioo0Axpo8tATAXQC+wswHiKgFwFRmtp0OoAqCFACEu8RCzLVT04mOdXSF0NlFGDU8swhm7/4wKso5w1rXaNkVAWZchWpk5nD3IIxvYDXWQdwWf7OsAb88Ng+VSAkUA9lXbWpFKE7GgTU0iAuCVYGPlgpoNW5PE+Z9+zKnwt9wQ3qwtKxMXBjycQdorSzNWLvW/H1qn4XmKtKvFci/yMarweMFIueCIGa+hJknGP8B+BuAGgDv9or4eABvE5GZ6CsUmZhlsejIpsd4ZXkSA8uSpumL2uNmdHSFcNfKcbgL9cI9o38MFVhe+xTuWX86Epub8PGTW/Hg9P9ME3EgBxEHhIiYuR5mz065ZTTq6kR2hSzop7cqZel4lZUiEyUez3TxmAUm8xVx7T3KMLsj0azzNWtE50LjWoH8A4hmLpcCltJ7hWsl+soiV+SEvgtinjAD9z89ElecfxBVo7vRtieCu1aOw9olzabXC2ag7kc1WLdJWKoz0YB7sARVaEMbqvAftffhhsXj0y8Cbky8txtETST85Y2N4nGt3F6zqOPx1LZYLN2y1o7ptJugG0O3ZTQ0iJxwM6zK4r0YEC6zxL26G/EImUWuhFxRXLxkfl44nenQsiuCmhkT07Y1r9+K6rGZFwqzfZ08L6+eKuFwKofcyvUgE2KZK8HJ4ATZ63jV+8ZKyK1EWfa55LpWq74n3/mOaBFgRN//pYjwvNcKM1c7EXGFwhJJ9adTA7NqdKbwmmWYaC4VQF5DYnYsAJk9VSoqgNra9IrB2lpzF8dwXf65letB9mZlgVKzwKITvMzYkOVna+X1MtzOLrEKupqJOCC2B8jFoio7FcVFzTjpQ0QABoTBLI/7te2JpLUeAYB1m8wrRjWXSiJhrrlteyQtBZKJTD/txo2pZk89PeJ3s6ZW8XjKN51r7rhZul0uudBeZ2zYldfr0fupDx82r4bNda2yddjlyXvU4MoLlJAr8sL1ONGYqHWZdTgM+tJUrN08MsNoZQY2vj0UdZfE0bJ+K5IvNKF1w1bMrI1j3aYoamZMRPjiqaiZMbFPxIGUFhsxzRUPhYAJJzkr9KirE72ujWhWtSx33E7czSxTJ9ZqWZm4sBRi6HZDgzyQbcwbNwZ9tSCnW2vN1ZL3qlDIA5SQK3LGaa0LABHU7K3OxOtbxe8yTrT44vUGRf//iw6a6t9Xz/87OraIAiIioGpMNx66vRXXXxJHNJpZM6MZenV1mfqiWfI72nUtBU6KiYuNU+y64OlzxzXRuukmZxkqesyyVYzCvWqV6JbodaWhdmKY3TaZrd/M9XHsmLgIurFWWf64XZ58gAqFlJArcsZxy2WL0W56+qz746OIH5RY5b0+9MEDzP3XI4clMsfEDUyi4d92or09M8tOb+iZfd9/+1oUL3X1thQ4d2J2Ig448/caU+L0vb2B1B2KlWVqVpZeKOE2IvPXh8Pm6/e65ausZH/ZsuwvmEWKmhCkyBnHyQWywp+BESGOyEwsmFkbx8rF6QOYEQqlLGLJMS2zW76YEezPwPV6EY8mxRQ12WadeJFu6BR9yqZZSmeRoSYEKVzHcXKBLEdct91oxNm6NWrMM1HaD1hb8na43ueowF3wioJss078LJ3X/uBakFrrDx+wv48SckXOOP7+yURUt93sLnrdpihO+LrErTEmin98MDMT5dZfmA+nsMqG8ZwCdsErCrIV5kJd7EqgglMKMxf835lnnsmK0mDtWuZYjJlI/Fy71mSnXe3Mf3qL+cU3U//+9JbY3kssxtzbvirtH5HkmL2vHQplPmdmbTs3r3+XE5vf5Ob17/Ki69rND6DwDkcnRoHXU1GRfqJUVPi/riwB0MQmmqp85Ar30JfbD4wAI4YC+w6K3weERXepREI8VjMuzcJuaBBtRsxO5AlVtAAABjlJREFURytXaUODqOuQpQSXujta4RA//fAu4nmJfjYoIS9BtMwUqxJqfbDSBFmQMpvK7IA3t1N4hdtl/z6hgp0Kb7EYEtFHMgk075S6KmXzBbJJ5+1v7miFQ0p4qASghFzhFg67F3JXt7SIqIT7/iv8psRPLiXkCndwmN63Mx6RFhH1x0w9RYEo8ZNL+cgV7uDQR173wxge35jpIw+Yq1Kh8AXlI1d4y5ioCGQO1BXwHD8y/feTYnj1Q/NAZ4m4KhUKXxjg9wIUJcSYqG0vkvp684r1EnFVKhS+oCxyRUEpcVelQuELyiJXFJy6OiXcCoWb5G2RE9EiItpGRH8hov/jxqIUCoVC4Zy8hJyILgLwVQBnMPNpAH7qyqoU/ZpS7m2kUHhBvq6V+QB+zMxHAYCZ9+S/JEV/xti+WysYApQ7RqGQka9r5SQAFxDRG0T0EhGdJduRiOYRURMRNe3duzfPl1X4jVdWs+OpQwqFog9bi5yINgIYa/LQkt7njwBwLoCzADxBRJ9lkyojZl4BYAUgCoLyWbTCX7y0mr2e+qVQlCK2FjkzX8LME0z+/RbADgBP9bbK/S8ASQAjvV60wl+8tJpLvLeRQuEJ+bpWngFwEQAQ0UkAIgDa812Uorjx0mou8d5GCoUn5CvkjwD4LBH9N4D1AOaYuVUUpYWXVrMqGFIosievrBVm7gYwy6W1KAKC12X2qmBIocgOVaKvyBplNSsUxYUq0VfkhLKaFYriQVnkCoVCEXCUkCsUCkXAUUKuUCgUAUcJuUKhUAQcJeQKhUIRcHwZvkxEewF0oPSqQEdCvaegUIrvqxTfE1Ca7yvX9xRj5lHGjb4IOQAQUZPZNOggo95TcCjF91WK7wkozffl9ntSrhWFQqEIOErIFQqFIuD4KeQrfHxtr1DvKTiU4vsqxfcElOb7cvU9+eYjVygUCoU7KNeKQqFQBBwl5AqFQhFwfBVyIlpERNuI6C9E9H/8XIvbENE/ERETUeBH3xHR0t6/01YiepqIhvu9plwhosuI6H+I6CMiusPv9bgBEZ1ARC8Q0fu936Vb/V6TWxBRmIjeIaL/8HstbkBEw4noyd7v0wdEdJ4bx/VNyInoIgBfBXAGM58G4Kd+rcVtiOgEAF8BUCojg58HMIGZJwLYDuBOn9eTE0QUBnA/gH8AcCqAmUR0qr+rcoUeAP/EzKdCDEJfWCLvCwBuBfCB34twkWUA/sjMJwM4Ay69Nz8t8vkAfszMRwGAmff4uBa3uRfA7QBKIpLMzM8xc0/vr68DGO/nevLgbAAfMfPfeqdbrYcwJgINM3/KzG/3/v8QhDiM83dV+UNE4wFcDuAhv9fiBkQ0DMCFAB4GxIQ1Zt7vxrH9FPKTAFxARG8Q0UtEdJaPa3ENIvoqgJ3M/K7fa/GIGwH8p9+LyJFxAD7W/b4DJSB4eoioGsBkAG/4uxJX+DmEQZT0eyEuUQNgL4BVve6ih4io0o0DezohiIg2Ahhr8tCS3tceAXEreBaAJ4jos0EY3mzzvu6CcKsECqv3xMy/7d1nCcRtfEMh16ZwBhENBvAbAN9l5oN+rycfiOgKAHuY+S0i+pLf63GJAQCmAFjEzG8Q0TIAdwD4324c2DOY+RLZY0Q0H8BTvcL9X0SUhGgks9fLNbmB7H0R0ekQV913iQgQLoi3iehsZt5VwCVmjdXfCgCI6JsArgBQG4SLrYSdAE7Q/T6+d1vgIaIyCBFvYOan/F6PC0wDcBURTQdQDmAoEa1l5iAPe98BYAcza3dLT0IIed746Vp5BsBFAEBEJwGIIOAdzpj5PWYezczVzFwN8YebUuwibgcRXQZxi3sVM3f6vZ48eBPAiURUQ0QRADMA/M7nNeUNCavhYQAfMPPP/F6PGzDzncw8vvd7NAPA5oCLOHp14GMi+nzvploA77txbD+HLz8C4BEi+m8A3QDmBNjSK3V+CWAggOd77zReZ+ab/F1S9jBzDxHdDOBZAGEAjzDzX3xelhtMAzAbwHtEtKV3213M3OjjmhTmLALQ0GtI/A3ADW4cVJXoKxQKRcBRlZ0KhUIRcJSQKxQKRcBRQq5QKBQBRwm5QqFQBBwl5AqFQhFwlJArFApFwFFCrlAoFAHn/wEKTUNsD1VhSAAAAABJRU5ErkJggg==\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    }
  ]
}